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PREFACE 


This  document  was  prepared  by  the  Institute  for  Defense  Analyses  (IDA)  for  the 
Office  of  the  Assistant  Secretary  of  Defense  (Production  and  Logistics)  under  a  task  entitled 
“Comparison  of  CAMS/REMIS  and  TICARRS.”  This  document  contains  comments  on 
IDA  Paper  P-2863,  “A  Comparison  of  Air  Force  Data  Systems,”  which  was  prepared 
under  the  same  task.  Paper  P-2863  is  IDA’s  final  report  on  an  evaluation  of  the  costs  and 
operational  effectiveness  of  two  automated  Air  Force  systems  for  maintenance  information. 
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i.  INTRODUCTION 


The  Institute  for  Defense  Analyses  (IDA)  conducted  a  study  of  the  costs  and 
operational  effectiveness  of  two  automated  systems  for  collecting  and  providing 
maintenance  and  related  information  to  the  Air  Force.  One  of  these  systems 
(CAMS/REMIS)  combines  the  Core  Automated  Maintenance  System  (CAMS)  and  the 
Reliability  and  Maintainability  Information  System  (REMIS).  The  other  is  the  Tactical 
Interim  CAMS  and  REMIS  Reporting  System  (TICARRS).  The  outcome  of  that 
comparison  is  documented  in  IDA  Paper  P-2863,  “A  Comparison  of  Air  Force  Data 
Systems.”  Before  the  release  of  that  paper,  interested  parties  were  given  the  opportunity  to 
comment  on  a  draft  version.  Members  of  the  IDA  study  team  met  with  the  interested  groups 
to  discuss  their  comments.  IDA  considered  the  information  obtained  from  the  meetings  in 
preparing  the  final  version  of  the  paper. 

The  sponsor  in  the  Office  of  the  Secretary  of  Defense  (OSD)  asked  that  comments 
and  responses  be  published  for  the  record.  This  document  therefore  contains  copies  of  the 
comments  provided  by  the  OSD,  the  Air  Force,  Litton  Computer  Services,  and  Dynamics 
Research  Corporation.  In  addition,  it  contains  IDA’s  responses  to  those  comments. 

This  document  was  prepared  with  the  expectation  that  readers  would  have  access  to 
the  draft  version  of  IDA  Paper  P-2863  (the  version  being  commented  upon). 
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II.  THE  OFFICE  OF  THE  SECRETARY  OF  DEFENSE 


A.  ASSISTANT  DEPUTY  UNDER  SECRETARY 

(MAINTENANCE  POLICY) 

OSD  Comments: 

A  copy  of  the  letter  from  Robert  T.  Mason,  Assistant  Deputy  Under  Secretary 
(Maintenance  Policy),  appears  on  pages  II-2  and  11-3  of  this  document. 

IDA  Response: 

1 .  Alternative  2  MAISRC:  We  agree  and  have  modified  the  report  to  reflect  this 
information. 

2.  Alternative  2  lines  of  code:  We  have  re-examined  our  estimates  and  provided 
additional  discussion  on  definitions  of  lines  of  code  in  the  report 

3 .  Alternative  1  data  accuracy  and  responsiveness;  We  have  attempted  to  include  the 
resources  required  to  improve  data  accuracy  and  responsiveness  for  Alternative  1; 
however,  the  inherent  system  architecture  of  CAMS/REMIS  works  against  their 
simple  resolution. 

4.  Impact  of  F-16  disconnect  from  TICARRS,  CAMS  data  base  quality;  We 
presented  information  on  data  quality  and  logistics  measures  before  and  after  the 
F-16  disconnect.  We  have  not  addressed  the  CAMS  data  base  quality  problems, 
because  we  did  not  structure  a  test  during  the  Operational  Assessment  to  assess 
them.  These  problems  were  reported  in  detail  by  DRC. 

5.  Two-level  maintenance:  Yes,  for  both  alternatives  we  have  included  resources  for 
modifications  to  support  two-level  maintenance. 

6.  RPC  costs:  We  have  changed  our  methodology  to  define  these  costs,  and  we  have 
provided  additional  detail  in  the  report. 

7 .  Review  of  historical  cost/performance:  We  have  considered  prior  performance  both 
in  estimating  costs  and  in  assessing  the  risk  of  the  alternatives. 
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OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 
WASHINGTON.  DC  20301-3000 


Institute  for  Defense  Analyses 
Attn:  Wayneurd  C.  Devers 
1801  N.  Beauregeurd  Street 
Alexandria/  VA  22311-1772 

Dear  Sir: 


Affected  OSD  offices  conducted  a  review  of  the  draft  of  IDA 

Paper  P-2863/  "A  Ccoiparison  of  Air  Force  Data  Systems.”  Inputs  eure 

summarized  beloW/  and  specific  coomients  are  attached. 

•  Alternative  #2  (TICARRS)  needs  to  account  for  MAISRC 
oversight /approval  in  terms  of  both  cost  and  schedule. 

•  Alternative  #2  costing  for  conqoarable  functionality  in  terms  of 
lines  of  new  code  and  cost  per  line  appears  low. 

•  Alternative  #1  cost  estimate  should  more  specifically  account  for 
in^roved  data  accuracy  eind  responsiveness. 

•  Expand  (if  possible)  ir^act  of  F-16  disconnect  from  direct 
TICARRS  input.  Expand  explanation  of  data  beise  quality  problems 
in  exporting  CAMS  data  to  TICARRS  92  at  Seymour  Johnson  AFB. 

•  Do  the  cost  estimates  for  both  alternatives  reflect  directly  the 
system  modifications  necessary  to  support  two- level  maintenemce? 

•  Alternative  one  is  extremely  sensitive  to  the  CAMS  marginal  costs 
at  the  RPCs.  Continue  efforts  to  further  define  costs /  identify 
sensitivity  of  conclusions  to/level  of  confidence  in  the  costs. 
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B .  DIRECTOR  FOR  PROGRAM  OVERSIGHT, 

ODASD(C3I  ACQUISITION) 

OSD  Comments: 

A  copy  of  the  memorandum  containing  the  comments  from  Col.  Johnnie  O. 
Rankin,  Director  for  Program  Oversight,  Office  of  the  Deputy  Assistant  Secretary  of 
Defense  (C3I  Acquisition)  appears  on  page  II-S  of  this  document 

IDA  Response: 

1 .  MAISRC  requirement:  We  agree  and  have  modified  the  report  to  reflect  this 
information. 

2.  TICARRS  enhancements:  We  have  done  additional  woik  on  our  estimates  of  hnes 
of  code  for  TICARRS  enhancements.  We  discuss  this  in  detail  in  the  report 
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COMMAND  CONTROL 
C  DM  MUN  1 C  A  T  )ON9 
AND  iNTCLLiOCNCC 


OFFICE  OF  -^HE  ASSISTANT  SECRETARY  OF  DEFENSE 

WASHINGTON.  DC  20301-3040 


July  21,  1993 


MEMORANDUM  FOR  DIRECTOR,  MAINTENANCE  POLICY,  ODUSD (LOGISTICS) 

SUBJECT:  Institute  for  Defense  Analvses  Review  Version  Report  "A 
COMPARISON  OF  AIR  FORCE  DATA  SYSTEMS" 


We  have  review  the  subject  report  and  would  like  the 
following  comments  included  as  part  of  the  OSD  comments  provided 

IDA: 


-  On  pages  Vll-l  and  VlI-16,  IDA  has  incorrectly  assumed 
that  TICARRS  would  not  require  OSD  MAISRC  Milestone  III  approval 
before  deployment  and  implementation.  Because  of  IDA's  projected 
life-cycle  costs,  to  be  incurred  between  1994  and  2003,  and  the 
existing  special  interest  in  this  program,  TICARRS  should  be 
considered  a  major  AIS  program.  In  turn,  it  will  be  subject  to 
MAISRC  Milestone  III  review  and  approval  and  the  requirements  of 
DoDI  8120.2.  For  this  reason,  IDA  needs  to  adjust  its  costing 
and  schedule  for  TICARRS  completion,  testing,  and  implementation. 
Major  exit  criteria  that  will  have  to  be  satisfied  before  the 
MAISRC  review  will  be  completion  of  system  development, 
developmental  and  operational  testing  to  demonstrate  that  user 
requirements  can  be  met,  economic  analysis  information  updated, 
and  an  independent  cost  evaluation  completed. 

-  On  page  VII-30,  the  draft  report  states  that  DRC 
estimates  it  will  take  30,000  lines  of  code  for  the  eight 
functional  enhancements  required  for  comparability  to  CAM3/REMIS 
enhanced  capabilities.  However,  for  software  maintenance 
purposes,  IDA  assumes  a  growth  of  150,000  lines  of  code.  On  pa 
Vli-21,  IDA  projects  $1,612,800  for  this  work  ($10.75  per  line 
code) .  Based  on  the  information  provided,  the  projected  lines 
new  code  required  for  having  comparable  functionality  seems  low 
and  the  cost  per  line  of  code  appears  very  optimistic. 

If  you  have  any  questions,  please  contact  my  action 
officer,  David  Dore,  at  (703)  746-7925. 


xnnie  O.  Rankin,  COI,  USA 
.rector  for  Program  Oversight 
0DASD(C3I  Acquisition) 


:  o  o 


C .  LEAD  CAMS/REMIS  MAISRC  ACTION  OFFICER 


OSD  Comments: 

A  copy  of  the  memorandum  from  David  A.  Dore,  Lead  CAMS/REMIS  MAISRC 
Action  Officer,  appears  on  page  II-7  of  this  document 

IDA  Response: 

We  have  revised  the  cost  and  schedule  estimates  in  the  report  to  include  a  MAISRC 
review  under  both  alternatives. 
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OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  DEFENSE 


WASHINGTON.  DC  20301-3040 


COMMAND.  CONTROt,. 
COMMUNICATION* 
ANO  MTCLUCCNCC 


July  30,  1993 

MEMORANDUM  FOR  DIRECTOR,  MAINTENANCE  POLICY,  ODUSD (LOGISTICS) 


SUBJECT:  Institute  for  Defense  Analyses  Review  Version  Report  on 
CAMS/REMIS  and  TICARRS  Comparison 

The  Institute  for  Defense  Analysis  (IDA)  has  provided  its 
draft  report  on  their  current  comparative  study  of  the  Air  Force 
Core  Automated  Maintenance  System/Reliability  and  Maintainability 
Information  System  (CAMS/REMIS)  and  the  Tactical  Interim  CAMS  and 
REMIS  (TICARRS)  automated  information  systems.  On  July  21,  1993 
COL  Rar.)cin  provided  cur  major  comments  on  the  draft.  This  is  to 
further  clarify  our  remarks  on  the  need  for  MAISRC  review. 

Whether  the  CAMS/REMIS  or  the  TICARRS  alternative  is 
selected,  a  MAISRC  Milestone  III  review  will  be  required  before 
either  alternative  can  provide  operational  support.  If  the 
CAMS/REMIS  alternative  is  selected,  the  expected  MAISRC  Milestone 
III  approval  should  be  a  conditional  approval  for  the  GCSAS 
portion  of  REMIS,  pending  completion  of  the  remaining  planned 
CAMS/REMIS  improvements  and  then  a  total  CAMS/REMIS  system 
operational  test/assessment.  If  the  TICARRS  alternative  is 
selected,  the  MAISRC  Milestone  III  review  would  require 
completion  of  a  full  operational  test,  by  an  independent  testing 
organization,  that  results  in  certification  that  the  validated 
CAMS/REMIS  Mission  Need  Statement  requirements  are  satisfied. 

If  you  have  any  questions  regarding  this  matter,  please 
contact  me  on  (703)  746-7925. 


David  A.  Dore 

Lead  CAMS/REMIS  MAISRC  Action  Officer 
Office  of  the  Deputy  Assistant  Secretary  of  Defense 

(C3I  Acquisition) 


D.  STAFF  ANALYST,  FORCE  STRUCTURE  AND  INFRASTRUCTURE 

COST  ANALYSIS  DIVISION 

OSD  Comments: 

A  copy  of  the  memorandum  from  Francis  L.  McDonald,  Staff  Analyst,  Force 
Structure  and  Infrastructure  Cost  Analysis  Division  appears  on  pages  0-9  and  II- 10  of  this 
document. 

IDA  Response: 

1 .  CAMS/REMIS  data  accuracy  and  responsiveness:  We  have  included  resources  in 
our  Alternative  1  cost  estimates  to  resolve  these  issues.  The  report  discusses  these 
resources.  We  have  also  considered  the  historical  record  of  problems  in 
CAMS/REMIS  in  evaluating  the  risks  of  each  alternative. 

2.  Quality  control  after  F-16  switch  to  CAMS:  We  have  provided  as  much 
information  as  we  have  on  this  issue  in  the  report.  We  do  not  have  sufficient 
information  to  conduct  a  more  formal  statistical  analysis. 

3.  Two-level  maintenance:  Yes,  for  both  alternatives  we  have  included  resources  for 
modifications  to  support  two-level  maintenance. 

4.  Marginal  costs  for  CAMS  at  RPCs:  We  have  examined  our  methodology  relative  to 
the  RPCs,  and  the  relevant  costs  are  reported. 

5 .  Level  of  confidence  of  cost  estimates:  We  have  assessed  technical  risks  given  the 
resources  estimated  for  the  alternatives  rather  than  assessing  the  cost  risks  in  detail. 
Obviously  we  could  reduce  technical  risk  by  allocating  more  resources  to  particular 
problems.  That  produces  feasible  but  not  realistic  solutions. 


OFFICE  OF  THE  SECRETARY  OF  DEFENSE 


WAfiHINGTON.  DC  20301-1000 


MEMORANDUM  FOR  DIRECTOR  OF  MAINTENANCE  POLICY 
SUBJECT:  ^  eport  on  CAMS/REM  IS  and  TICARRS 


f2llf  •  J(il  ,00' 


This  memorandmn  sutninarizes  my  assessment  of  the  IDA  study  that  compares 
CAMS/REMIS  with  TICARRS.  I  believe  that  the  study  is  generally  based  on  a  proper 
analytical  framework  but  needs  to  be  improved  in  the  following  areas: 


•  Given  the  overwhelming  accuracy  and  responsiveness  shortcomings  that 
have  been  ascribed  to  the  CAMS/REMIS  systems  in  this  study,  it  is  clear  that  the  system 
functional  compari.son  ba.seline  needs  to  explicitly  incorporate  the  requirements  for  timely, 
responsive,  and  accurate  information,  llte  internal  Air  Force  Program  Management 
Directives  (PMDs),  formalized  for  these  systems,  must  have  addressed  the  need  for  timely, 
responsive,  and  accurate  information  as  a  system  objective.  (The  prolonged  existence  of  these 
data  accuracy  and  responsiveness  problems  in  CAMS/REMIS  is  mindboggling  and  seriously 
questions  the  futility  of  management  efforts  and  analyses  based  on  data/information  from 
these  systems.)  The  cost  estimate  for  alternate  #1  (CAMS/REMIS)  in  this  study  must 
specifically  account  for  improving  data  accuracy  and  responsiveness.  What  efforts  are  being 
undertaken  to  accotnplish  this  objective?  How  were  these  efforts  costed  out?  What  level  of 
confidence  (uncertainty)  do  we  have  in  these  cost  estimates? 

•  The  study  needs  to  specifically  address  what  happened  from  a  quality 
control  standpoint  when  the  Air  Force  disconnected  the  F-16  CDS  and  utilized  CAMS  inputs 
for  TICARRS.  Additionally,  there  is  a  need  to  document  the  quality  of  the  database  that  was 
utilized  at  Seymour  Johnson  AFB  for  the  TICARRS-92  Operational  Assessment.  What  was 
the  error  rate  for  the  CAMS  data  entering  the  TICARRS?  Was  the  error  rate  in  the  database 
statistically  significant? 

•  As  the  Air  Force  transitions  from  a  three  level  to  a  two  level  maintenance 
concept,  the  requirement  for  an  integrated  management  infonnation  system  that  produces 
accurate  and  responsive  infonnation  with  real  time  access  becomes  very  important.  The 
alternatives  presented  in  this  study  appear  to  be  oriented  toward  system  comparisons  in  a 
three  level  maintenance  concept  environment.  The  system  functional  comparison  baseline  in 
a  two  level  maintenance  concept  environment  would  give  weight  to  the  fact  that 
CAMS/REMIS  are  es.sentially  "stovepipe"  type  data  systems  while  TICARRS  represents  a 
limited  but  integrated  management  information  system.  The  TICARRS  has  the  basic  inherent 
capabilities  but  its  weapon  system  coverage  and  functionality  (e.g.  base  supply  interface, 
engine  management,  etc.)  needs  to  be  expanded.  The  cost  estimates  for  alternative  #I  and  #2 
should  then  reflect  the  appropriate  system  modifications  that  are  required  to  accommodate  the 


management  visibility  requirements  associated  with  implementing  a  two  level  maintenance 
concept. 


•  The  documentation  for  alternative  #1  needs  to  highlight  the  methodology 
used  for  determining  the  marginal  costs  for  CAMS  at  Regional  Processing  Centers  (RPCs). 
What  level  of  confidence  do  we  have  in  this  methodology?  Do  changes  in  CAMS  recurring 
marginal  costs  have  a  major  impact  on  the  results  of  the  study?  If  so,  how  large  do  thses 
changes  have  to  be? 

•  One  needs  to  establish  a  level  of  confidence  (uncertainty)  associated  with 
the  cost  estimates  for  alternative  #1  and  #2.  An  explicit  historical  review  of  the  cost  and 
schedule  performance  for  prior  CAMS,  REMIS,  and  TICARRS  efforts  is  the  appropriate  basis 
for  this  effort.  (Has  there  been  a  consistent  pattern  and  experience  of  meeting  projected 
schedules  and  milestones?  Has  there  been  a  consistent  pattern  with  respect  to  the  actual  cost 
of  work  performed  versus  the  projected  cost  of  work  to  be  performed?  Has  the  deliverables 
been  available  for  immediate  operational  use  or  have  additional  modifications  or  reworks  been 
required  to  permit  effective  operational  use?)  The  study  should  disclose  this  past  history  to 
the  consumer  of  the  report.  Based  on  past  performance,  one  will  find  that  a  distinctly 
different  level  of  confidence  could  be  assigned  to  alternative  #1  (CAMS/REMIS 
enhancement)  than  for  alternative  #2  (TICARRS  enhancement). 


The  incorponuion  of  these  changes  .should  allow  this  study  to  provide  a  proper  basis 
for  detemining  the  most  appropriate  course  for  future  Air  Force  maintenance  management 
information  systems. 


Francis  L.  Me  Donald 
Staff  Analyst 

Force  Structure  and  Infrastructure  Cost  Analysis  Division 
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III.  THE  AIR  FORCE 


A.  FORWARDING  MEMORANDUM 

A  copy  of  the  forwarding  memorandum  from  John  M.  Gilligan,  Air  Force  Program 
Executive  Officer  for  Combat  Support  Systems,  appears  below. 


DEPARTMENT  OF  THE  AIR  FORCE 

AIR  FORCE  PROGRAM  EXECUTIVE  OFFICE 
WASHINGTON  DC 


JU1.3C1993 

MEMORANDUM  FOR:  ASD(C31) (Program  Oversight) 

Subject:  Comments  on  Draft  IDA  Comparison  of  CAMS/REMIS  and 
TICARRS 

Me  appreciate  the  opportunity  to  comment  on  the  draft  IDA 
report.  Our  Intent  was  to  limit  our  comments  to  areas  of 
fact,  required  assumptions,  and  the  development  of  conclusions 
from  these  facts  and  assumptions.  We  avoided  editiorlal 
comments  on  the  text  itself. 

Our  review  of  the  draft  report  uncovered  serious 
shortcomings  in  its  data,  analysis,  and  conclusions.  In  our 
judgement  these  shortcomings  are  so  severe  they  compromise  its 
credibility.  Without  resolution  of  these  identified  problems 
the  report  should  not  be  released. 

Our  comments  are  providtid  ..n  two  attachments.  The  first 
attachment  contains  comments  which  ma)ce  a  substantatlve 
difference  in  the  analysis  and  require  additional  effort  to 
Incorporate  into  the  final  report.  The  second  attachment 
contains  comparatively  minor  comments  which  would  improve  the 
report  but  would  not  require  additional  analysis. 

If  you  have  any  questions  on  these  comments  please 
contact  me.  I  am  ready  to  provide  any  needed  assistance  as 
IDA  moves  towards  finalizing  their  report. 


Air  Force  Program  Executive  Officer 
.for  Combat  Support  Systems 

2  Atch 

1 .  Substantative  Comments 

2 .  Other  Comments 
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B.  SUBSTANTIVE  COMMENTS 


Air  Force  Comments: 

A  copy  of  the  substantive  comments  from  the  Air  Force  appears  on  pages  III-8 
through  III- 18  of  this  document.  The  page  numbers  in  both  the  Air  Force  comments  and 
the  EDA  responses  that  follow  refer  to  page  numbers  in  the  review  version  of  EDA  Paper  P- 
2863. 

IDA  Response: 

1 .  Page  II-7,  second  paragraph;  We  have  removed  the  detailed  material  about  the 
systems’  functions  from  Chapter  II  and,  instead,  rely  on  Chapter  V  to  define  the 
capabilities  of  each  of  the  systems.  The  “bad  actor”  tracking  capability  is  addressed 
in  Chapter  V.  However,  it  should  be  noted  that  bad  actors  refer  to  part/serial 
numbers.  While  this  has  been  performed  at  (X)-ALC,  it  has  not  been  as  widely 
used  as  a  REMIS  function  as  it  has  in  TICARRS. 

2.  Page  11-12,  second  paragraph  (deEciencies  identified  in  the  Seymour  Johnson 
Operational  Assessment):  Again,  we  have  removed  the  detailed  material  about  the 
systems’  functions  from  Chapter  E.  Relative  to  this  comment,  our  assessment  in 
Chapter  V  takes  into  consideration  the  findings  from  the  Seymour  Johnson 
Operational  Assessment.  The  Operational  Assessment  identiEed  three  or  four 
serious  deficiencies.  The  SBSS  interface,  CEMS  interface,  and  so  on,  were  not 
part  of  the  assessment;  it  was  known  in  advance  that  TICARRS  could  not  comply 
with  Air  Force  requirements  in  this  area.  The  IDA  team  has  estimated  the  cost  of 
adding  these  functions  to  TICARRS. 

3 .  Page  B- 12,  third  paragraph:  We  agree  that  most,  rather  than  many,  of  the  aircraft 
types  supported  in  REMIS  are  not  now  supported  by  TICARRS.  We  have 
addressed  this  issue  in  detail  in  Chapter  V  of  the  report. 

4.  Page  V-1,  Erst  paragraph:  CAMS/REMIS  was  deployed  using  a  highly-structured 
acquisition  process.  Both  the  CAMS  segment  and  the  REMIS  segment  have 
numerous  documented  problems,  several  of  which  surfaced  immediately  after  the 
systems  were  deployed.  The  Air  Force  has  only  begun  to  address  user  problems 
with  CAMS,  and  it  has  not  addressed  the  lack  of  consistent  availability  of 
reliability  and  maintainability  statistics  in  the  PPS  module  of  REMIS.  SufEcient 
resources  have  not  been  programmed  into  its  current  budget  to  address  these 
problems.  We  agree  that  TICARRS  was  developed  using  a  substantially  less- 
structured  process.  The  government  and  the  contractor  had  considerable  freedom 
in  finding  ways  to  meet  requirements.  Even  though  it  was  less  structured,  the 
TICARRS  acquisition  process  had  mechanisms  for  ensuring  user  involvement;  it 
was  directed  by  an  Air  Force  Functional  Review  Board  with  representatives  from 


each  of  the  operating  and  supporting  commands.  The  approach  to  developing 
TICARRS  relied  largely  on  rapid  prototyping  concepts  that  allowed  for  gaining 
agreement  on  the  look  and  feel  of  the  interface  before  major  programming  efforts 
were  undertaken.  The  TICARRS  process  for  revising  and  enhancing  software  to 
meet  user  requests  is  also  simpler  than  that  for  CAMS/REMIS.  TICARRS  has 
addressed  the  needs  of  the  users  and  has  demonstrated  that  accurate  data  were 
being  placed  into  the  system  (under  oirect-entry  TICARRS)  and  that  data  can  be 
extracted  from  the  system.  We  acknowledge  the  comment,  and  the  report  has  been 
modified  to  reflect  this  information. 

5.  Page  V-13,  last  paragraph:  We  agree  that  “Bright  Flag”  is  a  single  command 
initiative  and  we  have  provided  costs  to  add  a  training  module  to  TICARRS.  Our 
draft  narrative  was  not  updated  to  reflect  the  requirement  to  provide  this  module  in 
TICARRS.  It  has  been  modified  for  the  final  version. 

6.  Page  V-15,  first  paragraph  under  “Conclusions  Concerning  Functionality”:  We 
agree  that  user-defined  requirements  should  be  the  basis  for  evaluating 
CAMS/REMIS  and  TICARRS  functionality.  In  making  this  assessment,  however, 
we  believed  that  it  was  appropriate  in  our  independent  assessment  to  address  user 
requirements  that  are  in  the  process  of  being  defined  by  the  Air  Force.  For 
example,  the  REMIS  ORD  is  now  being  re-written.  Possible  user  requirements 
include,  but  are  not  limited  to  deployment  capability  and  support  of  two-level 
maintenance.  IDA  believes  that  it  is  useful  to  assess  the  extent  to  which  functions 
are  carried  out  and  how  well  they  perform,  for  comparison  purposes. 

7 .  Page  V-15,  last  paragraph:  IDA  was  hesitant  to  assemble  Table  V- 1  because  of  the 
many  nuances  involved  in  the  performance  of  a  function.  The  text  points  out  that 
within  a  function,  different  systems  may  have  relative  strengths  and  weaknesses 
with  respect  to  component  features  of  the  function.  Assigning  superiority  to  one 
system  over  another,  except  in  a  few,  clearly  dominating  instances,  implies  too 
much  about  how  to  weigh  the  various  functions. 

8.  Pages  V-17  through  37:  IDA’s  assessment  of  the  “operating  characteristics”  for 
CAMS  was  limited  to  available  data,  most  of  which  is  from  the  current  SBLC 
architecture.  Our  assessment  does  address  where  we  believe  that  the  RPC 
approach  to  supporting  CAMS  could  provide  advantages;  however,  many  of  the 
problems  with  the  SBLC  architecture  will  continue  under  the  RPC  environment 

9.  Pages  V-21,  first  paragraph:  IDA  agrees  that  both  CAMS  and  TICARRS  will  be 
held  captive  to  the  base  communications  infrastructure.  Because  of  limits  on  the 
availability  of  data  for  the  communications  contribution  to  response  time,  we  have 
discounted  that  from  our  analysis.  However,  about  20  percent  of  the  bases  that  use 
CAMS  have  response-time  problems.  During  our  survey  of  CAMS  operations  at 
four  bases,  we  observed  cases  where  response  time  in  CAMS  appeared  excessive, 
and  there  is  substantial  anecdotal  evidence  from  our  interviews  with  CAMS  users 
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that  support  this  as  a  major  problem.  The  report  states  our  belief  that  the  situation 
would  improve  under  the  RPC  environment. 

10.  Page  V-26,  first  paragraph:  IDA  is  aware  of  the  response-time  requirement  in  the 
Functional  Description.  We  also  know  that  with  the  computer  updates  to  the 
SBLCs  and  RPCs  and  other  improvements  to  the  CAMS  system,  the  standard 
being  used  at  the  Standard  Systems  Center  to  measure  performance  is  7  seconds. 
We  believe  that  this  is,  in  fact,  the  current  objective  that  the  system’s  performance 
is  being  designed  to  and  measured  against. 

1 1 .  Page  V-33,  “Software  Process”  subsection:  We  do  not  agree  that  this  material  is 
superfluous  to  the  report;  nonetheless,  we  have  removed  it. 

12.  Page  V-33,  “Requirements  Management  “  section:  This  section  does  not  address 
the  Air  Force-defined  process  to  establish  requirements  within  the  acquisition 
process.  As  explained  in  the  text,  it  does  make  a  comparison  of  the  systems  to  the 
criteria  established  by  Software  Engineering  Institute  for  evaluating  organizations 
developing  information  systems.  We  have  removed  this  section  from  Chapter  V. 

13.  Page  V-42,  comparison  of  survey  result,:  We  agree,  and  the  report  has  been 
modified  to  reflect  this  position. 

14.  Page  V-43:  The  factors  were  based  on  conversations  with  users  at  Seymour 
Johnson  AFB. 

15.  Page  V-44,  paragraph  2:  We  agree,  and  the  report  has  been  modified  to  reflect  this 
concern. 

16.  Page  V-44,  paragraph  3:  We  do  not  understand  this  comment  in  the  context  of  the 
stated  paragraph  in  the  repxirt. 

17.  Page  V-55  through  V-65:  We  believe  that  this  subsection  on  adaptability  provides 
important  information  relative  to  the  capability  of  the  systems  to  adapt  to  the  future 
needs  of  the  Air  Force  and  emerging  weapon  system  and  information  system 
technology.  The  report  includes  sufficient  qualifiers  for  a  reader  to  understand  that 
this  is  IDA’S  view  of  the  future  based  on  information  obtained  from  the  Air  Force 
and  relevant  contractors.  As  a  result  of  internal  IDA  review,  we  have  restructured 
the  presentation  of  the  material:  the  portion  of  the  subsection  that  addresses  the 
future  Air  Force  has  been  moved  to  an  appendix  and  the  body  of  the  paper  contains 
a  summary  of  that  material. 

18.  Page  V-67,  second  paragraph:  We  retained  this  paragraph  in  the  paper.  We  do  not 
want  to  leave  a  reader  with  the  impression  that  maintenance  information  systems 
have  no  impact  on  logistics  or  operational  performance  of  a  weapon  system.  Our 
conclusion  was  a  statistical  one,  a  failure  to  reject  the  null  hypothesis.  Previous 
research  performed  on  the  CAMS  for  Airlift  system  supports  the  basic  hypothesis 
that  maintenance  information  systems  can  affect  logistics  and  operational 
performance  of  weapon  systems.  There  are  two  possibilities  as  to  the  effects  of  a 


in-4 


maintenance  information  system  that  our  analysis  does  not  have  the  capability  to 
address.  First,  the  effects  of  the  two  information  systems  logistics  and  operational 
performance  are  so  similar  that  we  cannot  differentiate  between  the  systems. 
Second,  the  effects  of  maintenance  information  systems  can  only  be  measured  for 
a  longer  period  of  time  than  analyzed  in  our  study.  The  tests  provided  here  only 
addressed  the  short-term  influences.  In  sum,  we  support  the  view  that  accurate 
maintenance  information  will  affect  the  logistics  and  operational  performance  of 
weapon  systems.  The  tests  provided  here  only  addressed  the  short-term 
influences.  The  report  has  been  modified  to  make  this  point  clearer. 

19.  Page  VI- 1,  “Areas  of  Enhancement  for  CAMS”,  first  paragraph;  We  agree  and 
have  revised  the  language  based  on  new  information  provided  by  the  Standard 
System  Center  on  SBLC  performance. 

20.  Page  VI-3,  last  paragraph:  We  agree  and  have  removed  the  requirement  for 
estimating  the  costs  of  these  improvements  from  both  systems. 

21.  Page  VI-4,  last  paragraph:  We  agree  and  have  revised  our  schedule  to  include  a 
competitive  procurement  and  MAISRC  III  decision. 

22.  Chapter  VII  overall;  We  agree.  Our  objective  is  to  provide  enough  detail  to  allow 
reconstruction  of  the  IDA  estimate.  Relative  to  estimates  for  software  cost,  we 
believe  that  in  many  cases  staff-years  represent  a  more  accurate  approach  to 
estimating  the  effort  than  detailed  lines-of-code  costing.  The  narrative  description 
provides  our  approach  to  estimating  costs;  however,  where  appropriate  and  where 
an  independent  assessment  of  costs  was  necessary,  we  have  relied  on  the  Software 
Production  Quality  Reliability  version  20  (SPQR/20)  software  cost-estimating 
model  to  estimate  the  costs  using  lines  of  code. 

23.  Page  VII- 1,  second  paragraph;  We  have  modified  our  schedule  to  accommodate 
the  requirements  for  a  MAISRC  as  specified  by  the  Air  Force. 

24.  Page  VII-2,  Non-recurring  Hardware  Costs;  We  agree  and  have  modified  the 
report  to  reflect  this  information. 

25.  Page  VII-5,  Non-recurring  Hardware  Costs;  We  have  recalculated  the  costs  based 
on  information  from  the  Standard  Systems  Center. 

26.  Page  VII-5,  Non-recurring  Software,  Application  Software,  Enhanced  Data 
Editing  and  Control;  We  disagree  and  have  done  an  independent  cost  estimate. 

27.  Page  VII-6,  Non-recurring  Software,  Application  Software,  System  Integration 
and  Test;  We  disagree  and  have  done  an  independent  cost  estimate. 

28.  Page  VII-7,  Recurring  Costs,  User  Support,  CAMS  Base  Representatives;  We 
agree  and  have  modified  the  report  to  reflect  this  information. 

29.  Page  VII-8,  Recurring  Costs,  Software  Maintenance,  System  Integration  and  Test; 
We  agree  and  have  modified  the  report  to  reflect  this  information. 
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30.  Page  VII-8,  Recurring  Costs,  Software  Maintenance,  Documentation:  We  used  a 
uniform  percentage  across  all  three  systems. 

31.  Page  VII- 11.  Non-Recurring  Costs,  Hardware:  We  agree  and  have  modified  the 
report  to  reflect  this  information. 

32.  Page  VII-11,  Non-Recurring  Costs,  Software,  Performance  Improvement:  We 
disagree  and  have  done  an  independent  cost  estimate. 

33.  Page  VII- 13,  Recurring  Costs,  Computer  Operations,  Tandem  Hardware  at  the 
ALCs,  Operational  Staff:  We  agree  and  have  modified  the  report  to  reflect  this 
information. 

34.  Page  VII- 14,  G&A  and  Profit:  We  agree  and  have  modified  the  report  to  reflect 
this  informadon. 

35.  Page  VII- 16,  first  paragraph  (both  comments):  We  agree  and  have  modified  the 
report  to  reflect  this  informadon. 

36.  Page  VII- 18,  Non-Recurring  Costs,  Hardware:  We  agree  and  have  modified  the 
report  to  reflect  this  informadon. 

37.  Page  VII- 19,  Non-Recurring  Costs,  Software,  Applicadon  Software,  Funcdonal 
Enhancements  (Provisions  for  F-22,  etc.):We  agree  and  have  modified  the  report 
to  reflect  this  informadon. 

38.  Page  VII-19,  Non-Recuning  Costs,  Software,  Applicadon  Software,  Funcdonal 
Enhancements,  Paragraph  2e(l),  (Lines  of  code):  We  disagree,  and  we  have 
provided  addidonal  discussion  of  lines  of  code  in  the  report. 

39.  Page  VII-22,  Non-Recurring  Costs,  Initializing  Database  for  Each  Weapon 
System:  We  discussed  the  difflcuides  of  inidalizing  the  data  base  for  REMIS, 
because  the  baseline  estimate  included  no  money  for  REMIS  for  this  purpose,  and 
it  seems  to  be  a  formidable  task.  We  have  included  three  times  as  much  money  for 
TICARRS  data  base  initialization. 

40.  Page  VII-25,  Non-recurring,  Communications,  Hardware:  We  agree  and  have 
modified  the  report  to  reflect  this  information. 

4 1 .  Page  VII- 26  through  27,  Unit  Activation:  We  believe  that  some  factors  associated 
with  unit  activation  will  benefit  from  experience  and  average  costs  will  be 
significantly  lower  than  those  presented  in  the  comment  (e.g.,  DIREPs).  We  have 
allocated  sufficient  resources  to  cover  these  activities. 

42.  Page  VII-28,  Recurring  Costs,  User  Support:  We  agree  and  have  modified  the 
report  to  reflect  this  information. 

43.  Page  VII-30,  Recurring  Costs,  Software,  Documentation  Materials:  We  agree  and 
have  modified  the  report  to  reflect  this  information. 


44.  Page  VII-33.  Table  VII- 10,  Element  2.2.3;  The  costs  are  different  because  of 
phasing  in  of  the  operations. 

45.  Page  VII-35,  last  paragraph:  We  disagree.  No  DoD  or  Air  Force  regulation 
automatically  prevents  the  continuation  of  the  MAISRC  process  for  REMIS  under 
Alternative  2. 


SUBSTANTIVE  COMMENTS 


This  attachment  contains  substantive  comments  on  the  draft 
IDA  comparison  of  CAMS/REMIS  and  TICARRS.  Incorporation  of 
these  comments  into  the  final  report  may  require  additional 
analysis  effort.  Comments  are  keyed  to  the  draft  report  by 
page  and  section. 

Z  Zatroduetion 
No  substantive  comments. 

ZZ  System  Descriptions 

A.  Core  Autosiated  Maintenance  System  (CAMS) 

No  substantive  comments. 

B.  Reliability  and  Maintainability  Maintenauice  System 
(REMZS) 

Page  II-7,  second  paragraph:  PPS  has  a  "bad  actor" 
tracking  capability  comparable  to  that  contained  within 
TlCARRS-92;  this  capability  has  been  demonstrated  at  OC- 
ALC,  Tinker  AFB,  OK.  Since  no  formal  requirements  for  "bad 
actor"  tracking  have  been  defined,  neither  system  can  be 
said  to  have  a  better  ability  to  meet  Air  Force  needs.  To 
the  extent  the  Air  Force  has  a  defined  need  for  "bad  actor" 
identification  CAMS/REMIS  supports  that  need. 

C.  Tactical  Interim  CAMS  and  REMZS  Reporting  System 
(TZCARRS) 

Page  11-12,  second  paragraph  (deficiencies  identified 
in  the  Seymour- Johnson  Operational  Assessment) ;  The 
Operational  Assessment  identified  serious  deficiencies  in 
each  of  the  seven  TICARRS  sxibsystems,  where  the  subsystem 
failed  to  support  critical  Air  Force  maintenance  practices, 
contained  functional  characteristics  which  caused  increased 
workload  on  maintenance  technicians,  or  both.  Existing  CAMS 
capabilities  in  maintenance-supply  jnterface,  comprehensive 
engine  management,  personnel,  and  shop  production  planning, 
scheduling,  and  control  were  not  replicated  in  TICARRS.  The 
Operational  Assessment  concluded  that  "As  ot*  8  May  1993, 
TICARRS-92  cannot  support  the  4th  Wing  as  a  stand  alone 
maintenance  data  collection  system  ...  I  base  this  solely  on 
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those  func’-ional  capabilities  which  were  demonstrated  during 
this  assessment.** 

Page  11-12,  third  paragraph:  Most,  rather  than  many, 
of  the  aircraft  types  supported  by  REMIS  are  not  now 
supported  by  TICARRS.  As  referenced  later  in  the  report, 
this  includes  about  45  aircraft  MDS,  9  missile  variants,  all 
aircraft  trainers,  all  aerospace  ground  support  equipment, 
and  all  conununications-electronics  equipment. 

XIZ  Evaluation  Considerations 
No  substantive  comments. 

rv  Sources  of  Information 
No  substantive  comments. 

V  Evaluation  of  Existing  Systems 

Page  V-l,  first  paragraph:  Given  the  structured 

nature  of  the  acquisition  system  under  which  CAMS/REMIS  is 
being  developed  and  fielded,  much  formal  documentation, 
analysis,  and  review  exists  of  this  program.  Since  TICARRS 
has  not  been  developed  under  the  same  rigorous  management 
structure  a  similar  level  of  study  does  not  exist  for  this 
system.  This  has  led  to  a  situation  where  a  great  deal  of 
formal  documentation,  highlighting  program  status, 
strengths,  and  weaknesses  exists  to  assist  in  the  evaluation 
of  CAMS/REMIS;  few  comparable  TICARRS  studie.s  are  available 
to  support  an  unbiased  evaluation. 

Page  V-13,  last  paragraph:  "Bright  Flag"  is  a  single 
command's  initiative.  In  order  to  meet  the  defined  Air 
Force  requirement  CAMS/REMIS  or  TICARRS-92  must  contain  the 
required  capability  to  support  all  Air  Force  units,  both  ACC 
and  non-ACC  assigned. 

Page  V-15,  first  paragraph  under  "Conclusions 
Concerning  Functionality":  The  formal  acquisition  system 
used  within  DoD  mandates  that  programs  be  developed  and 
tested  against  user-defined  requirements.  These  are  the 
requirements  which  CAMS/REMIS  or  TICARRS-92  must  support  and 
be  measured  against,  when  compared  to  these  requirements 
CAMS/REMIS  has  full  functionality.  Measuring  the  system 
against  any  other  set  of  requirements  is  inappropriate. 

Page  V-15,  last  paragraph:  A  review  of  the  verbal 
description  of  the  functionality  of  the  two  programs  leads 
to  the  following  assessment: 
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CAMS/RCMXS 

Neither 

TICARRS-92 

Superior 

Superior 

Superior 

Equipmenc  Inventory 

X 

Equipment  Status 

X 

Equipment  Utilization 

X 

Flight  Scheduling 

X 

HOC  Support 

X 

Debriefing 

X 

Maintenance  Scheduling 

and  Reporting 

X 

Maintenance-Supply 

Interface 

X{1) 

Comprehensive  Engine 

Management 

X(l) 

Cannibalization  Tracking 

and  Management 

X 

Configuration  Tracking 

and  Management 

X 

TCTO  Tracking  and  Hgt 

X 

Personnel  Tng,  Avail,  and  Kgt  X(l) 

Shop  Prod  Ping,  Sched, 

and  Control 

Xd) 

Mobilization  Planning 

X 

System  Deployability 

X 

(1)  No  TICARRS-92  Capability 


Pages  v-l7  through  37:  Throughout  the  "Operating 
Characteristics"  section  analysis  is  conducted  on  CAMS  using 
the  current  SBLC  architecture;  all  economic  analysis  in 
Chapter  VII  is  conducted  using  the  Regional  Processing 
Center  architecture.  Costs  can  not  be  estimated  against  one 
system  configuration  and  operating  performance  evaluated 
against  a  completely  different  configuration.  Consistency 
in  the  alternatives  evaluated  is  required  for  meaningful 
analysis. 

Page  V-21,  first  paragraph:  When  TICARRS-92  operates 
as  a  base-level  system  replicating  CAMS  functionality  it  is 
held  captive  to  the  ba.se  communications  infrastructure  as  is 
CAMS.  The  Operational  Assessment  demonstrated  that  the 
major  factor  in  system  response  time  is  the  base 
communications  infrastructure/  not  the  transaction 
processing  time  of  either  the  SBLC  for  CAMS  or  the  TICARRS- 
92  central  mainframe.  As  the  system  user  sees  end-to-end 
system  response  time  from  transmit  to  receipt  of 
transaction/  this  is  the  appropriate  measure  to  use.  If 
TICARRS-92  is  populated  throughout  the  Air  Force  its  overall 
response  time  on  a  base-by-base  level  win  be  compareUole  to 
that  of  CAMS  because  of  its  comparable  dependency  on  the 
existing  and  planned  communications  infrastructure. 

Page  V-26,  first  paragraph:  The  CAMS  requirement/  as 
specified  in  the  Functional  Description,  is  for  average 
response  times  of  10  seconds  or  less,  with  a  maximum 
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response  time  of  30  seconds.  Based  upon  the  stated  Air 
Force  requirement  the  response  times  are  acceptable. 
Additionally,  the  rationales  cited  in  the  paragraph  are 
speculative  and  unsupported. 

Page  v-33,  "Software  Process"  subsection:  This 
subsection  is  superfluous  to  the  report,  it  does  not 
contribute  to  the  comparison  of  CAMS/REMIS  and  TICARRS,  and 
is  not  referenced  elsewhere  in  the  report.  Recommend  its 
elimination  from  the  final  report. 

Page  v-33,  "Requirements  Management"  section:  The  Air 
Force  has  a  defined  process  used  to  establish  requirements 
within  the  acquisition  process.  TICARRS,  historically 
managed  as  an  element  of  the  F-16  weapon  system,  has  not 
come  under  this  process.  If  TICARRS  did  become  the  standard 
Air  Force  system  it  would  be  required  to  amend  the 
relatively  informal  process  used  to  determine  and  modify 
functionality  and  comply  with  the  more  restrictive 
procedures  of  the  acquisition  process. 

Page  v-42,  comparison  of  survey  results:  Of  the  32 
separate  comparisons  of  CAMS  and  TICARRS  in  Table  V-8,  26 
(81%)  indicated  a  preference  for  CAMS.  The  description  of 
these  results  do  not  reflect  this  preference;  instead,  the 
single  data  element  with  the  most  favorable  TICARRS  support 
is  emphasized.  It  would  be  more  correct  to  say  that  of 
those  expressing  a  preference,  in  mo.st  instances  a  weak 
preference  towards  CAMS  was  stated. 

Page  v-43,  last  paragraph  and  supporting  bullets  on 
page  V-43/44:  The  factors  cited  as  influencing  the  shift  in 
survey  results  are  all  speculative  and  unsupported.  It 
could  equally  be  speculated  that,  as  users  became  more 
familiar  with  TICARR.S,  they  recognized  that  CAMS  is  easier 
to  use. 

Page  v-44,  second  paragraph:  The  text  contradicts  the 
data  contained  in  Table  v-8.  It  would  be  more  correct  to 
say  that,  at  the  conclusion  of  the  test,  of  those  selecting 
a  preference  a  weak  leaning  towards  CAMS  was  noted. 

Page  v-44,  third  paragraph:  These  statements  are 
unsupported  and  speculative.  The  Operational  Assessment  was 
structured  with  the  full  support  and  participation  of  the 
TICARRS  development  contractor  and  were  based  on 
representations  the  contractor  had  made  concerning  TICARRS' 
performance. 
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Pages  V-55  through  v-65:  In  this  subsection  the  draft 
defines  major  environmental  factors  affecting  future 
logistics  systems;  maps  these  factors  into  Air  Force 
requirements;  develops  a  proposed  solution  to  meet  these 
requirements;  and  develops  a  program  to  achieve  this 
solution.  However,  the  multitude  of  potential  decision 
points  in  the  analysis  leads  to  a  wide  variety  of  potential 
programs,  only  one  of  which  is  addressed.  The  analysis  is 
extremely  sensitive  to  a  wide  variety  of  factors,  none  of 
which  are  covered.  Rather  than  develop  Air  Force  needs 
through  conjecture  the  analysis  would  be  better  served  using 
validated  requirements  and  then  justifying  any  excursions 
from  them. 

Page  v-67,  second  paragraph:  These  comments  are 
speculative  and  unsupported  by  the  preceding  analysis — the 
report  content  concluded  the  source  of  data,  CAMS  or 
TICARRS,  made  no  difference  in  weapon  system  performance. 

VI  Definition  of  Alternatives 

Page  VI- 1,  "Areas  of  Enhancement  for  CAMS",  first 
paragraph:  This  section  ignores  the  contribution  of  the 

base  level  communications  infrastructure  to  CAMS  (and 
TICARRS)  performance.  As  demonstrated  at  Seymour- Johnson 
AFB,  this  Infrastructure  plays  a  major  role  in  system 
responsiveness . 

Page  VI-3,  last  paragraph:  TICARRS  will  also  require 
expansion  to  support  the  F-22,  B-2  IMIS,  RAMPOD,  Job  Cost 
Accounting,  and  deployable  capability  identified  for 
CAMS/REMIS. 

Page  VI-4,  last  paragraph:  We  estimate  TICARRS  could 
not  be  fully  fielded  until  FY  98  vice  the  FY  96  used. 
Adoption  of  TICARRS  as  the  standard  Air  Force  system  would 
require  selection  of  a  contractor  through  a  competitive 
process;  development  of  an  ORD  and  TEMP;  conduct  of  an 
Operational  Test;  and  fielding  approval  through  a  MAISRC 
III  decision. 

VZI  Estimating  the  Costs  of  Alternatives 

Overall:  The  final  report  should  provide  sufficient 
detail  to  allow  reconstruction  of  the  estimate.  In 
P^^r't.icular ,  software  estimates  are  presented  in  person-years 
of  effort;  the  basis  for  these  estimates  (Lines  of  Code 
with  productivity  indices  or  other  estimating  methodologies) 
must  be  provided. 
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Page  VII-l,  second  paragraph:  The  thresholds  for 
MAISRC  review  of  a  program  are  anticipated  program  costs  in 
excess  of  $100m  in  FY90  dollars;  any  single  year  costs  in 
excess  of  $25M  in  FY90  dollars;  or  estimated  life  cycle 
cost'  in  excess  of  $300M  in  FY90  dollars  (DoD  8120.1).  In 
addition.  Automated  Information  System  modernization/ 
modification  efforts  are  also  subject  to  DoD  oversight  per 
DoD  8120.1.  As  such,  the  TICARRS  effort  envisioned  would 
require  DoD  oversight  and  review  prior  to  fielding.  Such 
oversight  entails  a  formally-conducted  Operational  Test  and 
Evaluation  Program  which  must  be  included  in  an  approved 
TEMP.  To  obtain  an  approved  TEMP  an  ORD  is  required.  This 
chain  of  activities  will  significantly  lengthen  the  assumed 
timelines  for  fielding  TICARRS. 

A.  Coat  Estlaatea  for  CAMS 

Page  vii-2.  Non-recurring  Hardware  Costs:  Funding  for 
the  Regional  Processing  Centers  will  be  expended  by  the  end 
of  FY94;  no  additional  funds  have  been  identified  beyond 
this  point.  The  last  base  migration  to  a  regional  center  is 
programmed  for  January  1995.  With  the  Regional  Processing 
Centers  so  close  to  completion  the  majority  of  the  costs 
have  been  incurred  and  are  not  subject  to  change. 

Page  VIl-5,  Non-recurring  Hardware  Costs:  The  draft 
IDA  analysis  used  a  very  simplistic  method  of  allocating  Air 
Force  Regional  Processing  Center  costs  to  CAMS.  The 
methodology  used  overstates  the  actual  marginal  costs  of 
CAMS/REMIS  to  the  Regional  Computer  Centers.  Specifically, 

•  Six  computer  systems  at  each  base  are  being  combined  into 
the  Regional  Centers,  not  only  the  SBLC  as  presupposed. 

•  The  methodology  apportioned  costs  based  on  the  CAMS 
portion  of  on-line  SBLC  transactions;  on-line 
transactions  account  for  only  30-35%  of  the  SBLC 
processor  load. 

•  The  total  costs  of  the  Regional  Processing  Center  effort 
were  used  to  determine  marginal  costs  attributable  to 
CAMS;  in  actuality,  only  a  portion  of  the  cost  of 
Regional  Processing  Centers  is  for  processors.  The  rest 
is  for  infrastructure  items — printers,  communications, 
and  so  on. 

>  All  funds  for  the  Regional  Processing  Centers  will  be 
expended  by  the  end  of  FY94;  the  ability  to  influence 
future  spending  is  limited  to  a  portion  substantially 
less  than  the  full  amount. 
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(Note:  A  meeting  on  the  costs  elements  of  the  Regional 
Processing  Centers  and  their  cost  apportionment  to  CAMS  was 
held  at  Gunter  Annex  to  Maxwell  AFB  on  July  23  to  provide 
additional  detailed  material  to  IDA  team  members.) 

Page  Vll-5,  Non-recurring,  Software,  Application 
Software,  Enhanced  Data  Editing  and  Control:  Internal  Air 
Force  estimates  for  this  effort  are  for  five  person-years  of 
effort. 

Page  VII-6,  Non-recurring,  Software,  Application 
Software,  System  Integration  and  Test;  This  effort  is 
currently  conducted  at  a  two  person  level  of  effort;  the 
rationale  for  the  increase  to  15  is  unsubstantiated. 

Page  Vll-7,  Recurring  Costs,  User  Support,  CAMS  Base 
Representatives:  These  individuals  will  not  fill  new 

positions.  Instead,  existing  positions  are  being  converted 
to  support  this  effort.  Estimates  are  that  50-75%  of  the 
positions  will  come  from  CAMS  Data  Base  Managers,  who  arc 
already  included  in  the  estimate.  Accordingly,  using  the 
lower  bound  of  50%,  only  half  of  these  costs  will  represent 
a  marginal  increase. 

Page  VII-8,  Recurring  Costs,  Software  Maintenance, 
System  Integration  and  Test:  This  effort  is  currently 
supported  by  five  full-time  and  two  half-time  positions,  for 
an  equivalent  level  of  effort  of  6  full-time  positions.  The 
projection  that  half  of  the  75  positions  support  CAMS  is  in 
error. 


Page  VII-8,  Recurring  Costs,  Software  Maintenance, 
Documentation:  This  effort  is  currently  supported  at  a  five 

person  level  of  effort,  not  the  two  cited  in  the  analysis. 

B.  Cost  Estimates  for  REMIS 

Page  VII-11,  Non-recarr5ng  Costs,  Hardware;  The 
estimate  should  reflect  the  20%  compound  annual  cost 
reduction  used  in  the  CAMS  and  TICARRS  estimates. 

Page  VII-11,  Non-recurring  Costs,  Software,  Performance 
Improvement:  The  estimate  should  be  for  15  staff  years  for 

one  year,  vice  the  three  estimated.  Initial  analysis  of  the 
required  performance  improvements  recommended  by  the 
performance  monitoring  team  indicated  the  effort  would  be 
accomplished  using  12,315  direct  manhours,  about  6.5 
manyears.  This  effort  would  be  accomplished  in  a  single 
year  to  more  rapidly  achieve  the  performance  enhancements. 
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Page  VII-13,  Recurring  Costs,  Computer  Operations, 
Tandem  Hardware  at  the  ALCs,  Operational  Staff:  These  are 
unmanned  centers  which  require  only  casual  support.  They 
will  be  operated  from  the  NCC  in  a  "lights  out"  mode.  User 
support  and  operation  of  the  computer  systems  are  co¬ 
functions  of  the  NCC  and  are  part  of  the  projected  manning 
for  NCC  operations.  No  additional  manning  is  required  for 
the  ALC  sites. 

Page  VII-14,  G£A  and  Profit:  G&A  and  profit  are 
applied  inconsistently  to  the  estimates  for  CAMS,  RCMIS,  and 
TICARRS.  Each  of  the  three  estimate.*;  as  done  by  IDA 
contains  a  mix  of  hardware,  government  personnel,  and 
contractor  personnel .  However, 

•  No  G&A  and  profit  charges  are  applied  to  CAMS. 

•  G&A  and  profit  charges  are  applied  to  all  estimated  REMIS 
costs. 

•  G&A  and  profit  charges  are  applied  to  the  majority  of 
estimated  TICARRS  costs;  the  reason  selected  costs  are 
not  covered  is  undefined. 

C.  Cost  Estimates  for  TXCARRS 

Page  VlI-16,  first  paragraph:  Assumed  schedules  must 
be  adjusted  to  account  for  ORD  and  TEMP  development. 
Operational  Test  and  Evaluation,  MAISRC  Milestone  III 
review,  and  the  required  competition  of  contractor  effort. 

Page  VII-16,  first  paragraph:  The  additional  marginal 
costs  associated  with  the  above  activities  must  be  included 
in  the  estimate.  As  a  point  of  comparison,  the  Operational 
Assessment  at  Seymour-Johnson  cost  about  S1.3M;  Operational 
Test  and  Evaluation  is  more  expensive  than  an  Operational 
Assessment . 

Page  vii-18.  Non-recurring  Costs,  Hardware:  a  back-up 
to  the  single  TICARRS  computer  center  will  be  required  and 
must  be  included  in  the  estimate.  Since  under  the 
centralized  approach  used  by  TICARRS  sortie  generation  for 
each  and  every  base,  both  training  and  wartime,  would  be 
dependent  on  the  central  computer  extensive  back-up 
provisions  are  mandatory. 

Page  vii-i9,  Non-recurring  Costs,  Software,  Application 
Software,  Functional  Enhancements:  Provisions  for  F-22,  B-2 
IMIS,  RAMPOD,  deployable  TICARRS,  and  other  system.*; 
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enhancements  must  be  included  to  preserve  comparability  with 
the  CAMS  estimated  costs. 

Page  VII-19/  Non-recurring  Costs,  Software,  Application 
Software,  Functional  Enhancements:  Paragraph  2e(l),  page  VII-30 
indicates  DRC  estimates  it  will  take  approximately  30,000  lines 
of  code  to  provide  the  eight  functional  enhancements.  Based  on 
the  LOG  provided  by  CSCs  within  the  CAMS  system,  this  appears  to 
be  a  significantly  underestimated.  The  following  table  shows  th 
unique  LOC  currently  used  by  CAMS  for  these  functions  (Note:  thi 
does  not  include  additional  LOC  used  for  the  PROCs  within  the 
programs . ) 


Supply 

CEMS 

Personnel 
Training 
Production  Mgt 
Automated  Forms 
PQDR 

Maint  Snapshot - 

Subtotal  (excl  C-E) 


TOTAL  incl  C-E) 


EXECUTABLE 

CODE 

29514 

84086 

27257 

55417 

2083 

19564 

11509 

242) _ 

231851 


259193 


EXECUTABLE  CODE 
WITH  REMARKS 
31718 
94305 
28534 
59344 
2254  (1) 
22630 
13220 
2440  (2) 
254445 


285122 


Note  1:  Only  NFSJ20,  screen  380.  The  production 
management  function  includes  more  than  just  screen  380.  However, 
only  this  one  program  was  counted  because  it  is  the  only  screen, 
referenced  in  the  IDA  report. 

Note  2:  Only  NFSP60. 

CAMS  currently  has  1.1  million  lines  of  code  (LOC)  for  all 
systems.  Excluding  the  285,000  LOC  for  these  eight  enhancements 
leaves  about  815,000  LOC  for  capabilities  similar  to  those  in 
TICARRS.  Paragraph  2b,  page  VI 1-20  indicates  that  TICARRS  has 
850,000  LOC.  This  statistic  indicates  that  TICARRS  uses  about 
the  same  number  of  LOC  as  CAMS  for  similar  functions.  Based  on 
this  fact  the  LOC  needed  for  the  eight  enhancements  appears  to  be 
between  250,000  and  285,000,  not  the  30,000  LOC  estimated  by  DRC. 

Page  VII-22,  Non-recurring  Costs,  Initializing  Database 
for  Each  Weapon  System:  The  comparable  cost  estimating 
section  for  REMIS  contained  a  description  of  the  difficulty 
associated  with  this  effort,  presumably  to  establish  the 
ris)c  involved.  Since  the  TICARRS  estimate  for  this  effort 
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Is  greater  than  that  for  REMIS  by  a  factor  of  four,  a 
similar  description  of  the  risks  involved  is  appropriate. 
Such  a  discussion  is  particularly  apt,  given  the  extensive 
problems  encountered  in  database  initialization  in  the 
Seymour-Johnson  AFB  Operational  Assessment. 

V  Page  VII-25,  Non-recurring,  Communications,  Hardware: 

Costs  associated  with  establishing  a  back-up  site  must  be 
included. 

Pages  vll-26  through  27,  "Unit  Activation":  DRC 
proposal  #P9320  for  the  Operational  Assessment  at  Seymour- 
Johnson  AFB  reflected  significantly  different  estimates. 

The  estimate  for  activating  a  unit,  based  on  experienced 
costs,  is 

•  Site  Surveys:  80  hours  at  $51 . 05/hour“S4 , 084 

•  Load  CAMS  Data:  320  hours  at  $60/hour-$l 9, 200 

•  User  Training:  1,388  hours  at  $52. 94 /hour  for  3 
units=$24, 494/unit 

•  Resolve  Difficulty  Reports:  The  proposal  for  DRC  staff 
at  Andover  MA  to  work.  DIREPs  and  related  problems  to 
support  the  Operational  Assessment  was  4,208.5  hours. 

The  report  states  150  DIREPs  were  processed  for  an 
average  of  28  hours  per  DIREP.  Presuming  that  DIREPs 
fall  from  50  per  unit  to  15  per  unit  during 
implementation,  the  average  DIREPs  per  unit  would  be 
32.5.  32.5  DIREPs/unit  X  28  hours/DIREP  X 

S60/hour«S54, 600/unit 

•  Short  Term  Site  Support:  At  Seymour-Johnson  the 
contractor  proposed  five  people  for  six  weeks.  1,200 
hours  X  $35/hour  for  3  units=S14, 000/unit 

•  Travel  and  Living  (Training) :  At  Seymour-Johnson  the 
contractor  proposed  13  personnel  for  a  20  day  period  to 
provide  training  for  3  units.  260  days  X  100/day  for  3 
units»$8, 667/unit. 

•  Travel  and  Living  (Short  Term  Support) :  The  contractor 
proposed  five  people  for  six  weeks  for  3  units.  38  days 
X  5  people  X  $100/day  for  3  units=$6, 333/unit . 

•  Travel  and  Living  (Site  Survey) :  DRC  proposed  two  people 
for  five  days  for  3  units.  10  days  X  $100/day  for  3 
units-$333/unit . 
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The  total  per  unit  based  on  DRC  proposed  costs  is 
131,711  per  unit  for  a  total  of  $34,771,704. 

Page  VII-28,  Recurring  Costs,  User  Support:  No  costs 
are  included  for  a  central  "help  desk"  to  accomplish  tasks 
and  provide  support  to  users  above  the  base  level.  The 
analysis  should  Indicate  either  such  user  support  is  not 
required  or  provide  an  estimate  of  its  cost. 

Page  VIl-30,  Recurring  Costs,  Software,  Documentation 
Materials:  Current  TICARRS  documentation  does  not  meet  DoD 
standards;  the  estimate  should  reflect  increased  costs  to 
upgrade  this  documentation  to  comply  with  requirements. 

D.  Cost  CoBpasison  of  Alternatives 

Page  VlI-33,  Table  VII-10,  Element  2.2.3:  The  1994 
CAMS  cost  for  Base  Communications  Operations  are  higher  than 
those  estimated  for  the  same  effort  in  the  CAMS-specific 
effort  in  1994.  The  differing  costs  need  to  be  reconciled. 

Page  VII-35,  last  paragraph:  Under  Alternative  2 
current  systems  programmed  to  be  replaced  by  CAMS/REMIS  will 
operate  for  up  to  an  additional  three  years,  compared  to 
Alternative  1.  The  costs  of  the  continued  operation  of 
these  current  systems  should  be  Included  in  the  analysis  of 
Alternative  2  as  a  marginal  cost. 
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C .  OTHER  COMMENTS 


Air  Force  Comments: 

A  copy  of  the  other  comments  from  the  Air  Force  appjears  beginning  on  page  III-24 
of  this  document.  The  page  numbers  in  both  the  Air  Force  comments  and  the  IDA 
responses  that  follow  refer  to  page  numbers  in  the  review  version  of  IDA  Paper  P-2863. 

IDA  Response: 

1.  Overall  (treatment  of  CAMS  and  REMIS  as  separate  programs);  Although 
CAMS/REMIS  is  the  Air  Force  maintenance  data  system,  CAMS  and  REMIS  were 
developed  separately.  Their  architecture,  hardware,  and  management  are  di fi  e  rent. 
Availability  of  cost  data  for  CAMS  and  REMIS  varied  greatly.  We  have  treated 
CAMS  and  REMIS  separately  in  the  descriptive  parts  of  the  report  and  for  general 
cost-estimating  purposes.  They  have  been  treated  as  one  entity  for  the  formal 
alternatives.  We  believe  this  treatment  is  appropriate,  and  the  report  has  been 
modified  to  clarify  this  stance. 

2 .  Overall  (observations):  It  has  been  necessary  to  use  our  own  direct  observations  to 
supplement  data  provided  by  others.  We  have  been  careful  to  label  perceptions  as 
such  in  the  text. 

3.  Page  I-l,  second  paragraph;  We  acknowledge  the  comment.  We  believe  that  the 
text  presents  accurately  the  rationale  for  the  study. 

4.  Page  II- 1,  first  paragraph;  We  agree,  and  the  report  has  been  modified  to  reflect 
this  information. 

5.  Page  II- 1,  second  paragraph,  first  bullet;  We  agree,  and  the  report  has  been 
modified  to  reflect  this  information. 

6.  Page  II-2,  third  bullet  from  top;  We  agree  and  have  modified  the  report  to  reflect 
this  information. 

7.  Page  II-2,  third  bullet  from  the  top;  We  agree  and  have  modified  the  report  to 
reflect  this  information. 

8 .  Page  II-3,  first  bullet  (both  comments):  Chapter  V  has  been  modified  to  reflect  this 
information. 

9.  Page  II-2  second  paragraph,  second  bullet;  As  previously  mentioned,  we  have 
removed  the  detailed  material  about  the  systems’  functions  from  Chapter  II.  We 
have  carefully  considered  the  requirements  for  TICARRS  to  replicate  this  function. 

10.  Page  n-3,  “System  Configuration  and  Management”,  second  paragraph;  We  have 
modified  the  report  to  clarify  this  issue. 
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1 1.  Page  II-4,  “Current  Status  and  Future  Plans”,  “Status”,  second  bullet;  We  agree 
and  have  modified  the  report  to  reflect  this  information. 

12.  Page  II-4,  “Current  Status  and  Future  Plans”,  “Status”,  third  bullet:  We  have 
modified  the  report  to  reflect  this  information. 

1 3.  Page  II-5,  paragraph  4:  We  have  modified  the  report  to  reflect  this  information. 

14.  Page  II-6  first  paragraph:  Again,  we  have  removed  the  detailed  material  about  the 
systems’  functions  from  Chapter  0,  relying  instead  on  Chapter  V  to  describe  each 
of  the  system’s  capabilities. 

15.  Page  II-8,  “System  Configuration  and  Management”:  The  discussion  here  is 
intentionally  brief. 

1 6.  Page  II-9,  second  bullet  at  top  of  page:  We  agree,  and  the  report  clearly  states  the 
Air  Force  point. 

17.  Page  II-9,  “Functional  Overview”,  third  bullet:  TICARRS  has  a  deployable  system 
that  was  used  for  the  F-1 17  aircraft  in  the  Persian  Gulf  War. 

18.  Page  II-9,  “Functional  Overview”,  second  paragraph:  We  acknowledge  the 
comment 

19.  Page  n-9,  last  paragraph,  and  page  II- 10,  first  three  paragraphs:  We  have  revised 
Chapter  II  so  that  it  is  a  more  general  overview  of  the  system’s  functions.  When 
describing  a  system’s  functions,  we  do  not  mean  to  imply  that  they  are  unique  to 
that  system. 

20.  Page  11-12,  last  paragraph:  DRC’s  analyses  of  the  work  needed  to  expand 
TICARRS  have  been  independently  reviewed  by  IDA,  and  we  have  developed  an 
independent  cost  estimate  in  the  report 

21.  Page  III- 10,  “The  Organization  of  Weapons  System  Management”:  We  have 
modified  the  report  to  reflect  this  information. 

22.  Page  IV-9,  first  paragraph:  We  have  done  an  independent  categorization  of  the 
DIREPs  and  FDWWs.  We  have  included  a  fuller  discussion  in  the  final  report 

23.  Page  IV-9,  last  paragraph:  It  was  unclear  how  serious  many  of  the  FDWWs  were, 
especially  those  that  reported  data  and  communications  problems.  In  fact,  many  of 
the  FDWWs  were  “fixed”  through  the  DIREP  process  during  the  assessment  (we 
know  that  this  may  not  have  been  the  correct  action  to  take,  but  the  fact  is  that  the 
problems  could  not  have  taken  substantial  resources  to  fix  if  they  were  completed 
so  quickly). 

24.  Page  V-2,  “Background”,  sub-paragraph  (3):  The  report  has  been  modified  to 
reflect  this  information. 

25.  Page  V-3,  Table  V-1:  IDA  understands  the  claim  regarding  narrative  data  in 
REMIS.  There  has  been  no  demonstration  that  the  narratives  can  be  retrieved.  Bad 
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actors  are  identified  by  part/serial  number,  not  WUC.  Regarding  cannibalization 
actions,  we  agree  and  the  report  has  been  modified  to  reflect  this  information. 
GCSAS  is  not  now  fielded  but  other  changes  have  been  made  to  the  table.  The  text 
and  table  have  been  revised  to  include  cannibalization  tracking. 

26.  Page  V-5,  last  paragraph;  The  report  has  been  modified  to  reflect  this  information. 

27.  Page  V-6,  second  paragraph:  The  report  has  been  modified  to  reflect  this 
information. 

28.  Page  V-9,  second  paragraph;  We  understand  but  can  give  no  credit  for  planned 
development. 

29.  Page  V-9,  third  paragraph;  The  report  has  been  modified  to  reflect  this 
information. 

30.  Page  V-10,  first  paragraph:  The  report  has  been  modified  to  reflect  this 
information. 

3 1 .  Page  V- 14,  “Mobilization  Planning”,  second  paragraph;  We  agree,  and  the  report 
has  been  modified  to  reflect  this  information. 

32.  Page  V-14,  “System  Deployability”:  We  agree;  however,  a  deployable  system  was 
used  for  the  F- 1 17s  in  the  Persian  Gulf  War. 

33.  Page  V-13,  third  paragraph:  In  the  final  report,  we  have  treated  engines  with  the 
GEMS  interface  and  communications-electronics  equipment  as  part  of  scope. 

34.  Page  V-17,  second  paragraph:  Given  the  low  usage  levels  of  REMIS  and  the 
results  of  our  own  observations,  we  believe  the  text  is  appropriate.  The  importance 
of  narratives  is  addressed  elsewhere  in  the  chapter. 

35.  Page  V-19,  first  bullet:  We  have  expanded  our  discussion  of  this  issue  in  the  final 
report  to  include  management  decisions  that  impact  on  availability. 

36.  Page  V-19,  last  paragraph:  See  note  34,  above.  The  expanded  treatment  should 
clarify  the  issues. 

37.  Page  V-20,  fourth  paragraph:  We  do  not  believe  “frequently”  implies  universality. 

38.  Page  V-26,  last  paragraph:  We  disagree.  Subsequent  analyses  substantiate  our 
language. 

39.  Page  V-27,  first  paragraph:  We  agree  and  have  modified  the  report  to  reflect  this 
information. 

40.  Page  V-30,  Table  V-5:  While  TICARRS  currently  shares  a  computer  with  other 
applications,  it  would  have  a  dedicated  computer  system  if  it  were  expanded  to 
cover  the  entire  Air  Force. 

41.  Page  V-30,  last  paragraph:  We  do  not  believe  the  text  implies  that  CAMS’s 
operational  effectiveness  is  wholly  dependent  on  base-level  ADP  personnel.  The 
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fact  remains  that,  under  the  current  system,  these  personnel  have  great 
responsibility. 

42.  Page  V-31,  “Shared/Dedicated”,  first  paragraph;  We  have  modified  the  text  to 
remove  tlie  speculative  statements. 

43.  Page  V-34,  “Software  Project  Tracking  and  Oversight”:  Noted. 

44.  Page  V-37,  second  paragraph;  Noted. 

45.  Page  V-38,  “Observations/Discussion  with  Users”,  second  paragraph,  sixth  bullet; 
While  training  may  have  been  rated  good,  users  complained  to  the  IDA  team  about 
the  quality  of  on-line  help.  Opinions  of  the  effectiveness  of  training  may  change 
weeks  after  the  class,  as  trainees  try  to  implement  what  they  have  learned  in  the 
field.  We  also  met  many  people  who  were  given  disks  to  access  REMIS  with  no 
training. 

46.  Page  V-38.  second  paragraph,  second  bullet  under  MAJCOMS;  Noted. 

47.  Page  V-39,  fourth  bullet:  As  previously  noted,  opinions  of  the  effectiveness  of 
training  may  change  weeks  after  the  class,  as  trainees  try  to  implement  what  they 
have  learned  in  the  field. 

48.  Page  V-39,  first  paragraph:  The  IDA  study  team  found  no  contractor — Rockwell, 
Lockheed  Georgia,  Lockheed-Fort  Worth,  or  McDonnell-Douglas — who  preferred 
using  on-line  REMIS  to  receiving  D056  tapes.  A  list  of  our  contacts  will  be 
provided  to  the  REMIS  PMO. 

49.  Page  V-46:  This  may  result  in  some  improvement,  but  the  capability  has  not  been 
demonstrated. 

50.  Page  V-47,  second  paragraph:  We  agree  and  have  modified  the  report  to  reflect  this 
information. 

5 1 .  Page  V-47,  third  paragraph:  We  agree  and  have  modified  the  report  to  reflect  this 
information. 

52.  Page  V-47,  third  paragraph:  We  agree  and  have  modified  the  report  to  reflect  this 
information. 

53.  Page  V-47  through  V-48,  last  paragraph/first  paragraph:  Noted.  We  stand  by  our 
assessment. 

54.  Page  V-48,  “Coronet  Deuce  Experience”:  Noted.  Users  have  indicated  to  us  that 
the  difficulty  of  using  CAMS  means  that  maintainers  sometimes  do  not  document 
their  actions  in  CAMS.  This  is  a  likely  cause  of  some  of  the  inaccuracy. 

55.  Page  V-49,  “Data  Loss  Problems”:  Noted. 

56.  Page  V-53,  “Integrity  and  Security  of  Data  Input”  subsection:  We  have  modified 
and  condensed  this  discussion. 


ni-22 


57.  Faj^e  V-53,  last  paragraph.  We  agree  and  have  modified  the  report  to  rellect  this 
information.  We  have  also  expanded  our  discussion  of  security. 

58.  Page  V-54,  first  paragraph;  Noted. 

j9.  Overall  Comment  on  Wording:  We  have  modified  the  report  to  reflect  this 
information. 
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OTHER  COMMENTS 


This  attachment  contains  other  comments  on  the  draft  IDA 
comparison  of  CAMS/REMIS  and  TICARRS.  These  comments  are 
comparatively  minor  ones  which  would  improve  the  report  but 
not  require  additional  analysis  to  incorporate  into  the 
final  version.  Comments  are  keyed  to  the  draft  report  by 
page  and  section. 

Overall 

Throughout  the  report  CAMS  and  REMIS  are  treated  as 
separate  programs.  There  is  a  singJe  Air  Force  program, 
CAMS/REMIS.  The  final  report  should  reflect  this  title; 
where  necessary  to  identify  functionality  or  costs  to  a 
selected  element  of  CAMS/REMIS  the  terms  "CAMS  segment"  or 
"REMIS  segment"  should  be  used. 

Throughout  the  report  much  use  is  made  of 
"observations",  "perceptions",  "beliefs"  and  "attributions" 
as  facts  and  conclusions.  While  observations,  perceptions, 
and  beliefs  are  data  used  to  develop  facts  and  conclusions, 
they  are  not  in  themselves  facts  and  conclusions  and  should 
not  be  presented  as  such. 

Z  Introduction 

Page  I-l,  second  paragraph:  The  CAMS  segment  also 
supports  NATO  AWACS  and  the  Royal  Netherlands  Air  Force. 

Page  I“l,  last  paragraph:  The  GAO-identi fied 
deficiencies  were  in  fact  surfaced  by  the  program  office; 
solutions  to  the  problems  are  well  underway  and  in  some 
cases  completed.  The  solutions  were  reviewed  and  accepted 
by  the  MAI SRC. 

II  System  Descriptions 

Page  II-l,  first  paragraph:  The  word  "currently" 
should  be  removed  from  the  first  sentence.  CAMS  is  the 
standard  Air  Force  base-level  maintenance  information 
system,  and  no  plans  exist  to  provide  this  functionality 
through  other  means. 

Page  II-l,  second  paragraph,  first  bullet:  The  Job 
Data  Documentation  (JOO)  subsystem  is  more  than  an  automated 
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349.  In  conjunction  with  the  Maintenance  Events  subsystem, 
it  provides  a  production  management  system  for  maintenance 
activities.  CAMS  is  geared  toward  production  management 
whereas  TICARRS  is  geared  toward  data  collection.  The 
Maintenance  Events  subsystem  provides  the  user  the 
capability  to  create,  schedule,  change,  defer,  reschedule 
and  cancel  events  and  work  center  events.  The  subsystem 
provides  management  products  to  aid  the  user  in  forecasting 
and  monitoring  inspection  and  time  change  requirements.  A 
distinct  advantage  the  subsystem  provides  is  the  work  center 
event.  This  allows  the  control  and  monitoring  of  work  flow, 
especially  where  multiple  shops  support  a  maintenance  event. 

Page  II-2,  third  bullet  from  top:  The  Interim  Direct 
Line  Reporting  (ILDR)  is  not  part  of  CAMS. 

Page  II-2,  third  bullet  from  the  top:  Additional  CAMS 
functions  added  since  1985  not  mentioned  are 

•  Automatic  Test  Equipment  Reporting  System  (ATERS) .  This 
subsystem  provides  on-line  access  to  a  database 
containing  organiration,  equipment,  status,  and 
utilization  data  for  assigned  test  equipment.  It  also 
keeps  track  of  the  capability  of  various  test  stations  to 
test  particular  parts.  This  permits  the  Avionics 
Intermediate  Shop  to  update  the  appropriate  data  in  the 
database  in  an  on-line  mode  as  events  occur  which  affect 
assigned  test  equipment. 

•  Product  Quality  Deficiency  Reporting  (PQDR)  Subsystem. 

The  PQDR  subsystem  reports  known  or  suspected 
deficiencies  for  equipment,  weapon  systems,  or  related 
components  and  records  exhibit  disposition  instructions 
and  data.  This  subsystem  has  an  interface  to  INFOCEN 
that  allows  approved  PQDRs  to  be  transmitted  directly  to 
INFOCEN  via  CAMS. 

Page  II-3,  first  bullet:  An  additional  bullet  is 
needed  to  specify  that  CAMS  has  the  capability  to  deploy  a 
suite  of  communications  hardware  to  maintain  connectivity  to 
home  station.  This  suite  will  enable  the  users  to  maintain 
connectivity  to  home  station  as  long  as  some  source  of 
communications  (dial-up,  DDN,  etc.)  is  available.  This 
equipment  is  available  via  standard  Air  Force  contract. 
Additionally,  the  capability  to  utilize  mobile  terminals 
using  radio  frequency  modems  has  also  been  added  to  the 
standard  Air  Force  contract. 
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Page  11-3/  first  bullet:  An  additional  bullet  is 
needed  to  specify  that  CAMS  incorporates  a  Communications- 
Electronics  Subsystem.  This  subsystem  provides  on-line 
access  that  permits  job  control  to  update  appropriate  data 
as  events  occur  which  affect  assigned  equipment  and 
missions.  It  also  provides  methods  for  reporting  inventory 
gains  and  losses,  mission  gains  and  losses,  and  maintains 
status  and  utilization. 

Page  II-2,  second  paragraph,  second  bullet:  Automated 
Forms  is  a  separate  subsystem  of  CAMS  and  should  not  be 
included  with  Automated  Debriefing.  The ’ Automated  Forms 
subsystem  provides  the  capability  to  Identify  discrepancies 
to  be  included  in  the  automated  781  series  forms.  The 
subsystem  retrieves  aircraft  maintenance  data  from  CAMS  and 
uses  it  to  prefill  the  781  series  forms. 

Page  II-3,  "System  Configuration  and  Management", 
second  paragraph:  When  lines  of  code  are  used  throughout 
the  report  compiled  or  uncompiled  should  be  specified. 

Page  II-4,  "Current  Status  and  Future  Plans",  "Status", 
second  bullet:  Follow-on  CEMS  interface  in  in-work. . 

Page  II-4,  "Current  Status  and  Future  Plans",  "Status", 
third  bullet:  The  deployable  CAMS  was  used  in  Desert  Storm. 

Page  Il-S,  paragraph  4:  Throughout  the  draft  report 
the  respective  roles  of  the  Data  Base  Administrator  (DBA) 
and  Data  Base  Manager  (DBM)  arc  intermixed.  These  two  roles 
are  separate  and  distinct.  DBAs  work  within  the  SBLC 
computer  center,  and  will  be  incorporated  into  the  Regional 
Processing  Centers.  DBMs  are  individuals  within  the  . 
maintenance  organizations  who  are  responsible  for  the 
actions  required  to  maintain  the  CAMS/REMIS  capability 
within  their  assigned  units.  As  an  example,  the  DBA  is 
responsible  for  loading  a  new  software  release  on  the  SBLC; 
the  DBM  is  responsible  for  notifying  affected  personnel  of 
the  impact  of  any  changes  on  their  use  of  the  system  (new 
procedures,  screens,  capabilities)  and  when  the  changes  will 
become  effective.  DBMs  currently  accomplish  tasks  beyond 
those  of  the  TICARRS  site  representative,  such  as  repairing 
pointer  errors  in  the  data  base,  or  developing  and  executing 
Query  Language  Program  inquiries. 
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At  the  unit  level  some  type  of  system  support  will  be 
needed/  by  either  CAMS/REMIS  or  TICARRS.  Eliminating  the 
DBAs  in  the  base  computer  system  will  not  affect  this  need. 

Page  11-6/  first  paragraph:  This  description  of  REMIS 
functions  should  also  include  Inventory  Control  through 
maintaining  the  Air  Force  Master  Inventory/  Inventory 
Assignment/  and  Inventory  Possession;  weapon  system  status, 
both  current  and  historical/  for  aircraft/  missiles/ 
trainers,  automated  test  equipment/  and  communications- 
electronics  equipment;  utilization  information  such  as 
actual  and  allocated  flying  hourS/  actual  sorties,  landings, 
and  full  stop  landings. 

Page  11-8,  "System  Configuration  and  Management":  A 
description  of  the  communications  architecture  supporting 
the  REMIS  segment  should  be  provided. 

Page  11-9,  second  bullet  at  top  of  page:  Contractors 
have  had  difficulty  obtaining  access  to  REMIS  because  of 
security  policy  issues,  not  because  of  system  architecture 
or  hardware  Issues  unique  to  REMIS.  Similar  problems  will 
be  experienced  by  any  automated  system  until  the  appropriate 
policy  has.  been  defined. 

Page  11-9,  “Functional  Overview",  third  bullet: 

TICARRS  does  not  provide  a  deployable  system.  It  does 
support  a  deployable  communications  capability,  in  the  same 
manner  that  CAMS  or  REMIS  can  be  accessed  world-wide  with 
appropriate  communications  links. 

Page  II-9,  "Functional  Overview",  second  paragraph: 

Only  two  of  the  five  Air  Logistics  Centers  currently  use 
TICARRS . 

Page  II-9,  last  paragraph,  and  page  11-10,  first  three 
paragraphs:  These  capabilities  are  also  resident  in 
CAMS/REMIS.  No  differentiation  is  provided  between  features 
common  with  CAMS/REMIS  and  any  unique  TICARRS  functionality; 
the  implication  in  this  section  as  written  is  only  TICARRS 
supports  the  listed  capabilities. 

Page  11-12,  last  paragraph:  DRC  effort  to  analyze  the 
work  needed  to  expand  TICARRS  has  been  accomplished 
independently  of  the  Air  Force  and  has  not  been  validated  by 
any  outside  organization. 
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2ZZ  KvalMtion  Considarations 

Page  111-10/  "The  Organization  of  Weapons  System 
Management"/  first  paragraph:  The  Air  Force  has  adopted, 
not  is  planning  to  adopt,  IWSM. 

ZV  Sources  of  Information 

Page  IV-9,  first  paragraph:  A  total  of  416  FDWWs  and 
149  DIREPs  were  developed  in  the  Seymour- Johnson  AFB 
Operational  Assessment.  These  FDWWs  and  DIREPs  were  broken 
out  into  six  categories,  with  DRC  DIREPs' directly  linked  to 
resolving  Identified  deficiencies.  The  breakout  was 

Software  Database  Training  Comnunications  Security  Other 


Number 

FDWWs  155  92  80  32  12  45 

Number 

Associated 

DIREPs  .  90  32  17  0  7  2 

As  noted  in  the  final  report  of  the  Operational 
Assessment,  Functional  Disconnects  identified  160  different 
software  related  problems. 

Page  IV- 9,  last  paragraph:  The  text  requires 
modification  to  reflect  the  seriousness  of  the  FDWWs  and 
DIREPs  written  during  the  Operational  Assessment  and  the 
software  modifications  made  in  response.  As  a  point  of 
fact,  the  Air  Force  Operational  Test  and  Evaluation  Center 
(AFOTEC)  does  not  allow  software  changes  during  an 
Operational  Test;  .  if  the  Operational  Assessment  had  been  a 
formal  Operational  Test  TICARRS-92  would  most  likely  have 
been  evaluated  as  not  operationally  effective/suitable. 

V  Evaluation  of  Existing  Systems 

Page  V-2,  "Background",  sub-paragraph  (3) :  Each  system 
was  based  on  different  requirements,  distinct  from  perceived 
needs.  TICARRS  was  developed  by  individual  weapons  systems 
program  offices  to  provide  certain  logistics  tracking 
capabilities  for  a  limited  set  of  items;  CAMS/REMIS  was 


Attachment  2 


ni-28 


developed  to  meet  Air  Force  wide  maintenance  and  logistics 
needs  established  through  validated  requirements. 

Page  V-3,  Table  V-l;  The  following  corrections  are 
needed: 

•  REMIS  does  receive  and  maintain  narrative  data  on 
debriefing  failure  history  and  corrective  action  under 
DON. 

•  REMIS  does  contain  serialized  part  maintenance  history 
and  provides  bad  actor  identification  by  WUC. 

•  REMIS/  under  PPS,  tracks  and  outputs  all  cannibalization 
actions. 

•  TCTO  management  requires  both  TCTO  Master  Information  and 
TCTO  Status.  REMIS  GCSAS  Supports  both  of  these 
functions;  TICARRS  supports  only  TCTO  Status. 

•  REMIS  supports  cannibalization  tracking. 

Page  V-5/  last  paragraph:  CAMS  supports  automated  test 
stations  by  providing  availability  data  through  the 
Automated  Test  Equipment  Reporting  Subsystem  (ATERS) ;  that 
function  was  not  used  at  Seymour* Johnson. 

Page  V-6/  second  paragraph:  Errors  pushed  back  to  CAMS 
from  REMIS  for  correction  are  not  dropped  after  an  interval 
of  time. 

Page  V-9/  second  paragraph:  Base-level  users  do  have 
access  to  REMIS  today;  they  do  not  have  access  through  CAMS 
terminals/  although  that  capability  is  being  developed. 

Page  v-9,  third  paragraph:  The  first  CAMS  increment, 

IB,  was  fielded  at  Dyess  AFB,  TX  in  1985  to  support  the  B-1. 
No  REMIS  capability  existed  at  that  time,  forcing  the  B-1 
contractor  to  extract  CAMS  data  in  its  own  data  base. 

Page  V-lO,  first  paragraph:  The  portions  of  the 
maintenance  community  were  positive  about  TICARRS* 
capabilities  should  be  specified. 

page  V-14,  "Mobilization  Planning:,  second  paragraph: 
The  interface  with  COMPES  was  determined  not  to  be  cost 
effective  and  was  deleted  from  CAMS  requirements.  CAMS  has 
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the  capability  to  identify  personnel  to  a  given  Mobility 
Group  Number.  Once  this  is  accomplished  CAMS  personnel  and 
training  records  are  provided  to  the  mobility  office.  There 
is  no  requirement  for  an  automated  interface  between  CAMS 
and  COMPES.  It  was  determined  that  the  manual  process  does 
support  the  COMPES  requirement. 

page  V-14,  "System  Deployability":  No  requirement  has 
been  defined;  until  this  occurs  neither  CAMS/REMIS  nor 
TICARRS  has  sufficient  direction  to  develop  a  robust 
deployable  capability. 

Page  V-13,  third  paragraph:  Engines  and 
communicatlons-electronics  equipment  will  have  to  be  added 
in  addition  to  missiles  and  simulators. 

Page  V-17,  second  paragraph:  The  implicit  conclusion 
that  TICARRS  is  significantly  superior  to  CAMS/REMIS  because 
it  provides  "important  narrative  Information"  is  unsupported 
by  the  previous  analysis. 

Page  v-19,  first  bullet:  As  a  general  rule  the  SBLCs 
are  required  to  be  available  24  hour.s  a  day,  seven  days  a 
week;  however,  on  an  exception  basis,  individual  bases  have 
made  the  management  decision  to  close  down  the  computer 
center  on  evenings  and  weekends.  Use  of  the  data  in  the 
draft  report  would  require  a  base-by-base  analysis  to 
determine  the  cause  of  the  availability  rates.  Examples  of 
bases  with  planned  availability  rates  below  80%  are 

•  Travis  AFB,  CA:  5  days/week,  0715-1730 

•  Goodfellow  AFB  TX:  5  days/week,  0730-2400 

•  Edwards  AFB  CA:  5  days/weck,  0730-1630 

•  Lowrey  AFB  CO  (base  closing} :  5  days/week,  0500-2300 

•  Shaw  AFB  NC:  5  days/week,  0630-1700,  plus  1  UTA/month 
0630-1700 

•  Bergstrom  AFB  TX  (base  closing) :  5  days/week,  0700-2200 

•  Chanute  AFB  IL  (base  closing) :  5  days/week,  0600-2400 

In  addition,  bases  supporting  strategic  airlift  missions  use 
the  G081  system  (CAMS  for  Airlift)  rather  than  the  CAMS. 
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Examples  of  strategic  airlift  mission  bases  with  planned 
system  availability  less  than  80%  are 

•  Travis  AFB  CA:  5  days/week,  0735-1730 

•  Dover  AFB  DE:  5  days/week^  0600-2400  plus  3  UTAs/month 
0600-2400 

Page  V-19,  last  paragraph:  The  reasons  advanced  for 
the  SBLC  availability  rate  are  speculative  and  not  supported 
by  data  within  the  draft  report. 

Page  V-20,  fourth  paragraph:  The  DC?  has  been 
identified  as  the  cause  of  poor  response  times  at  one  base; 
no  data  exists  as  to  the  universality  of  this  problem.  The 
word  "frequently**  is  speculative  and  unsupported. 

Page  V-26i  last  paragraph:  The  last  sentence  is 
speculative  and  unsupported;  it  expresses  conjecture  with 
no  analysis  or  fact. 


Page  V-27,  first  paragraph:  The  Air  Force  regional 
processing  centers  are  sized  at  16  megaword  (64  MB)  not  48 
MB. 


Page  V-30,  Table  v-5:  TICARRS,  as  mentioned  in  the 
text/  operates  in  a  shared  and  not  a  dedicated  environment. 

Page  V-30,  last  paragraph:  CAMS  operational 
effectiveness  depends  on  many  more  factors  than  the  base- 
level  ADP  personnel.  The  capabilities  of  the  base 
communications  infrastructure,  the  adequacy  of  user 
training,  and  management  procedures  within  the  maintenance 
complex  are  all  factors  which  have  a  major  impact  on 
operational  effectiveness  and  which  are  beyond  the  control 
of  the  ADP  function. 

Page  V-31,  "Shared/Dedicated",  first  paragraph;  The 
evidence  referenced  is  not  specified;  as  it  stands  the 
conclusion  is  unsupported  and  speculative. 

Page  V-24,  "Software  Project  Tracking  and  Oversight": 
The  CAMS  program  office  has  complied  with  all  applicable 
government  program  management  and  oversight  practices  and 
procedures,  as  they  have  evolved  since  the  program's 


Attachment  2 


in-31 


inception  a  decade  ago.  The  cost  accounting  practices  used 
do  not  supported  the  types  on  marginal  costing  required  by 
this  analysis;  they  were  never  developed  to  and  no 
requirement  has  ever  existed.  As  the  program  complies  with 
Defense  Business  Operating  Fund  required  practices  such 
information  will  be  available  In  the  future. 

Page  V-37,  second  paragraph:  As  referenced  elsewhere 
in  the  report/  corrective  action  to  remedy  editing  problems 
between  CAMS  and  REMIS  segments  Is  underway. 

Page  V-38/  "Observations/DiscussionS  with  Users", 
second  paragraph,  sixth  bullet:  To  date,  REMISTALK  training 
has  trained  151  students.  The  students  have  consistently 
rated  the  Instructor,  course  materials,  and  overall  course 
content  above  average  to  excellent  on  the  REMISTALK 
critiques.  The  record  shows  that  95%  of  the  students 
consistently  rated  all  areas  above  average. 

Page  V-38,  "Observations/Discussions  with  Users", 
second  paragraph,  second  bullet  under  "MAJCOMs":  Data  is 
downloaded  daily  by  REMIS  users.  An  explanation  of  the 
statement  is  required  to  determine  if  the  statement  refers 
to  communications,  LAN  or  other  unidentified  problems. 

Page  V-39,  fourth  bullet:  Litton  Computer  Systems  has 
developed  and  conducted  three  REMIS  courses  for  the  Air 
Force  between  1987  and  1993.  To  date,  REMIS  end-user 
training  has  trained  over  1,666  students.  Students  have 
cited  the  instructors  as  outstanding,  as  rated  the  course 
content  as  excellent.  Six  of  the  students  have  been 
contractor  personnel,  five  of  the  six  rated  all  areas  and 
the  CAI  as  a  "5"  or  "6"  on  the  student  critiques. 

Page  v-39,  first  paragraph,  "Contractors  have  the 
following  opinion":  During  October  1992  through  April  1993 
more  than  a  dozen  contractors  who  receive  DOS 6  data  on  tape 
were  contacted.  Each  was  offered  the  option  of  continuing 
to  receive  product  performance  data  on  tapes,  logging-on  to 
REMIS  to  view  the  data,  or  having  the  data  sent  via  DDN  to 
their  computers.  None  of  the  contractors  elected  to 
continue  to  receive  data  on  tapes. 

Page  V-46,  CAMS/REMIS  currently  has  software  in  testing 
which  enables  a  CAMS  user  to  query  the  REMIS  database  in  on- 
ii**®  real  time  mode.  This  is  the  first  step  in  providing 
CAMS  base  level  users  with  fleet-wide  data  validation  at  the 
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time  the  data  is  initially  entered  into  CAMS.  This 
capability  is  a  viable  alternative  to  a  single  centralized 
system  architecture. 

Page  V-47,  second  paragraph:  The  difference  methods 
used  to  round  flying  hours  in  EIMSURS  has  been  corrected. 
REMIS  and  CAMS  now  round  flying  hours  using  the  same 
algorithms . 

Page  v-47,  third  paragraph:  No  edits  have  been 
softened  in  either  CAMS  or  REMIS.  Doing  so  would  adversely 
effect  the  accuracy  and  integrity  of  the  data. 

Page  V-47#  third  paragraph:  Manual  loading  of  CJ\MS 
data  was  completed  in  June  1993  as  planned. 

Page  v-47  through  V-48#  last  paragraph/first  paragraph: 
Procedures  are  currently  being  defined  and  implemented  to 
ensure  synchronization  of  both  systems.  The  assessment  that 
regionalization  will  make  it  more  difficult  is  based  on  the 
assumption  that  it  will  reduce  the  number  of  data  base 
managers#  while  in  fact#  regionalization  does  not  effect  the 
number  of  DBMs.  The  functions  of  the  data  base  manager 
remain  unchanged.  End  of  month  reports,  data  base  fixes#  as 
well  as  generally  helping  the  CAMS  user  will  remain 
activities  of  the  data  base  manager  at  any  given  base.  The 
bulk  of  the  error  correction  from  REMIS  goes  to  and  should 
be  corrected  by  the  user  or  originator  of  that  transaction. 
The  data  base  manager  becomes  involved  only  when  necessary 
to  correct  a  serial  number#  organization  record,  or  similar 
element . 

Page  V-48,  "Coronet  Deuce  Experience":  Without  . 
supporting  information  the  true  cause  of  the  problem  can  not 
be  identified.  There  have  been  two  potential  causes 
proposed.  First,  the  TICARRS  data  transfer  file  from  base 
level  CAMS  may  not  have  been  sent  and  the  NRTS  actions 
picked  up.  This  is  unlikely  because  the  data  should  have 
been  sent  on  the  next  TICARRS  transmission.  In  addition, 
the  data  base  manager  should  have  been  notified  if  file  was 
not  received.  Second,  and  most  likely#  the  NRTS  action  was 
never  documented  in  the  CAMS  data  base.  During  initial 
Coronet  Deuce  testing  at  Hill  AFB,  this  was  a  common 
problem.  Coronet  Deuce  test  managers  estabJished  procedures 
to  validate  the  NRTS  action  before  an  LRU  wa.s  accepted  for 
shipment.  Coronet  Deuce  is  in  prototype  stages  and  as  with 
any  test  project  problems  must  be  identified  and  resolved. 
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Page  V-49,  "Data  Loss  Problems":  The  problems 
identified  by  Lockheed  have  been  corrected  or  will  be 
corrected  shortly.  One  fix  was  released  in  July  (9307),  one 
is  scheduled  for  the  November  (9311)  release,  and  the 
remainder  are  scheduled  to  be  released  with  GCSAS  (9401) . 

Page  V-53,  "Integrity  and  Security  of  Data  Input" 
subsection:  This  section  is  based  on  formal  reviews  of  SBLC 
and  REMIS  security  procedures  and  capabilities.  No  similar 
studies  have  been  conducted  on  TICARRS;  the  text  is 
misleading  in  not  recognizing  the  lack  of  a  comparable 
TICARRS  review. 

Page  V-53,  last  paragraph:  The  cited  AFAA  audit  report 
referenced  accessing  communications  processors;  there  is  a 
significant  distinction  between  accessing  a  communications 
processor  and  a  computer  system.  CAMS  has  user 
identification  and  password  protection  to  prevent 
unauthorized  access  to  the  required  security  level. 
Additionally,  the  cited  problems  with  "overstating  the 
quantity  of  parts  on  hand"  references  the  Standard  Base 
Supply  System;  CAMS  does  not  contain  the  capability  to 
modify  SBSS  supply  records.  With  the  lack  of  a  comparable 
review  of  TICARRS,  its  adequacy  under  equivalent  conditions 
is  unknown. 

Page  v-54/  first  paragraph:  The  REMIS  GCSAS  software 
was  modified  in  January  1993  to  provide  user  access 
restriction  by  Equipment  Designator  (id  KC-135  vs  C-141)  for 
all  approved  configuration  data.  This  modification  is 
scheduled  to  be  fielded  prior  to  the  first  weapon  system 
being  implemented  within  GCSAS. 

VZ  Definition  of  Alternatives 

No  additional  comments. 

VIZ  Zstimating  the  Costs  of  Alternatives 

IDA  correctly  accomplished  an  independent  estimate  of 
the  CAMS/REMIS  and  TICARRS  costs.  However,  in  some  cases 
the  language  used  to  describe  the  estimating  methodology 
reflects  uncritical  acceptance  of  DRC  estimates.  For 
example 

•  Page  VII-18:  "Based  upon  data  collected  by  DRC.  .  ." 
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•  Page  viI-20:  "DRC  has  provided  us  with  a  formal  impact 
analysis.  . 

•  Page  Vll-21:  .  .wc  are  using  the  DRC  estimates  of 

staff  hours." 

•  Page  VII»2€:  ".  .  .based  upon  a  written  description 

provided  by  DRC." 

•  Page  >^11-26:  "DRC  reported  that  the  CAMS  data  at 
Seymour- Johnson  were  extremely  error-prone.  . 

•  Page  VIl-27:  ORC  reported  that  these  DIREPs  averaged  20 
hours  apiece  to  research,  resolve,  retest.  .  .  " 

•  Page  VII-30:  "DRC  estimates  an  additional  30.000  lines 
of  code  for  the  eight  functional  enhancements." 

The  text  should  be  revised  to  reflect  IDA  estimates  of  these 

tasks,  using  DRC-supplled  data  as  one  element  of  the 

analysis. 
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IV.  LITTON  COMPUTER  SERVICES 


A .  FORWARDING  LETTER 

A  copy  of  the  forwarding  letter  from  Thomas  F.  King,  REMIS  Program  Director, 
Litton  Computer  Services  (LCS),  appears  below. 


Litton _ _ 

Computer  Services 


26  July  1993 


mx 


Mr.  Maynard  C.  Davars 
INSTITUTE  FOR  DEFENSE  ANALYSES 
1801  North  Baauragard  Straat 
Alaxandria,  Virginia  aaail'-lTYa 

Daar  Mr.  Davars: 

1  aa  forwarding  tha  final  varsion  of  our  coasMnts  on  your 
Raviaw  Varsion  of  IDA  Papar  P-2S63,  A  OMPARISOM  OF  AIR  FORCE 
DATA  SYSTEMS,  JUNE  1993. 

Bacauaa  of  tha  axtraaaly  short  tins  which  ws  wars  allowad 
to  raviaw  and  cosaant  on  this  raport,  tha  attached  rasponsa  doas 
not  nacassarily  include  all  of  tha  conents  which  wa  night  have 
nada,  given  sufficient  tina. 

Me  would  like  vary  such  to  continue  our  raviaw  and  provide 
you  with  additional  connents  and,  further  support  and  clarifica¬ 
tion  for  coanants  being  provided  at  this  tine,  which  nay  be 
helpful  to  you  as  you  prepare  to  present  tha  final  varsion  to 
the  U.S.  Congress. 


Sincerely, 
LITTON  COMPUTE 


SERVICES 


-xnonMiB  r.  ivxn^,  \ 

REMIS  Program  DiracMn 


TFK/jd 

TFK93.023 
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B.  COMMENTS 


LCS  Comments: 

A  copy  of  the  comments  from  LCS  appears  on  pages  IV-9  through  IV-41  of  this 
document.  Copies  of  the  various  attachments  referred  to  in  the  comments  appear  as 
follows:  Attachment  1  begins  on  page  rV-43;  Attachment  2.  on  page  rV-45;  Attachment  3  is 
on  page  IV-65;  Attachment  4  begins  on  page  IV-67;  Attachment  5  is  on  page  IV-69; 
Attachment  6  is  on  page  rV-71;  Attachment  7  begins  on  page  rV-73;  Attachment  8  begins 
on  page  IV-81;  and  Attachment  9  is  on  page  IV-85.  The  page  numbers  in  both  the  LCS 
comments  and  the  IDA  responses  that  follow  refer  to  page  numbers  in  the  review  version 
of  IDA  Paper  P-2863. 

IDA  Response: 

1 .  Page  I-l  and  1-2;  We  acknowledge  the  comment  regarding  the  GAO.  We  believe 
that  the  text  presents  accurately  the  rationale  for  the  study.  The  base-level  studies 
of  CAMS  are  described  in  the  GAO  report.  The  results  of  the  Operational 
Assessment  of  EIMSURS  and  PPS  are  not  yet  available,  and  we  have  noted  that  in 
the  report. 

2.  Page  II- 1,  Para  A.l:  Chapter  n  has  been  revised  and  presents  only  brief 
descriptions  of  the  systems.  More  detail  on  each  system  has  been  included  in 
Chapter  V. 

3.  Page  II-l,  Para  A.l,  Page  II-6,  Para  B.l,  Page  11-9,  Para  C.l:  Again,  Chapter  II 
has  been  revised  to  give  brief  descriptions  of  the  systems,  and  details  about  the 
systems  have  been  included  in  Chapter  V. 

4.  Page  II-2,  Bullet  3:  This  issue  is  treated  more  fully  in  Chapter  V. 

5 .  Page  II-3,  Para  2, 2nd  Subpara:  Lines  of  code  can  indeed  be  a  misleading  measure 
of  the  size  of  a  system.  We  have  included  REMIS  lines  of  code  in  the  report  and 
re-examined  our  analysis  of  this  issue. 

6.  Page  II-2  and  11-3;  The  CAMS  ATERS  function  has  been  covered  as  part  of  the 
scope  in  the  report  PQDR  has  been  treated  more  fiilly  in  Chapter  V. 

7.  Page  II-4,  Para  A.3.b,  Page  II-8,  Para  B.3.b,  and  Page  11-12,  Para  C.3.b: 
Coverage  of  these  issues  have  been  removed  from  Chapter  U.  They  are  treated 
more  fully  in  Chapter  V. 

8.  Page  n-S,  Para  3:  We  agree,  and  Uie  report  has  been  modiEed  to  reflect  this 
information. 

9.  Page  II-6,  Para  B.l,  1st  Subpara:  REMIS  functions  have  been  treated  more  fully 
in  Ch^ter  V.  It  is  not  yet  fully  operational. 
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10.  Page  11-6,  Para  B.l,  3rd  Subpara:  REMIS  functions  have  been  treated  more  fully 
in  Chapter  V. 

1 1.  Page  II-7,  Para  B.l:  Chapter  V  has  been  modified  to  provide  a  more  detailed 
explanation  of  these  systems. 

1 2.  Page  n-7,  Para  B.  1 :  A  more  complete  description  of  PPS  is  contained  in  Chapter 
V.  IDA  was  not  able  to  confirm  the  bad  actor  identification  or  serial  number 
tracking  capabilities  of  REMIS. 

13.  Page  II-8  and  II-9,  Para  3.B  and  3.C:  We  acknowledge  the  comment.  That 
material  has  been  removed  from  Chapter  II,  and  user  concerns  are  dealt  with  in 
Chapter  V. 

14.  Page  II-9,  Second  Bullet,  Top  of  Page:  The  material  has  been  removed  from  the 
chapter. 

15.  Page  II-9,  Para  C.l:  We  agree,  and  the  report  has  been  modified  to  reflect  this 
information. 

1 6.  Page  II- 10,  Para  C.  1 :  We  agree,  and  the  information  has  been  deleted. 

17.  Page  n-lO,  Para  3,  Last  sentence:  We  agree,  and  the  information  has  been  deleted. 

18.  Page  II- 1 1,  Para  C.2:  Parts  of  SDS  have  been  incorporated  into  TICARRS  92 
(AMOC). 

1 9.  Page  H- 12,  Para  B:  This  material  has  been  deleted  from  Chapter  n. 

20.  Page  11-12,  Para  C:  Information  from  DRC  was  used  as  a  starting  point  to  arrive  at 
independent  IDA  estimates. 

21.  Page  11-12,  Para  II.C.3.b:  The  discussion  of  perceived  problems  has  been 
removed  from  Chapter  11. 

22.  Page  IV-2,  Para  rV.B.2:  We  did  visit  the  F-1 17A  program  office,  and  we  received 
input  from  the  F-1 17A  community  during  our  visit  to  Holloman  AFB. 

23.  Page  V-3,  Table  V-1:  For  reporting  purposes,  we  believe  that  inventory 
possession  aud  assignment  can  be  combined.  We  have  changed  the  text  to  reflect 
narrative  information  that  was  available.  Bad  actor  identification  by  WUC  is  not 
sufficient  Bad  actors  are  part/serial  numbers.  The  table  and  text  have  been  revised 
to  include  cannibalization  and  TCTO  status  for  PPS. 

24.  Page  V-3,  Table  V-1,  a:  The  equipment  issue  was  addressed  in  the  previous 
comment.  TICARRS  requirements  for  these  functions  have  been  addressed  in 
software  changes  and  costs.  The  other  issues  here  are  a  matter  of  scope,  which  has 
been  addressed  and  estimated  for  TICARRS. 

25.  Page  V-3,  Table  V-1,  b:  This  issue  has  been  covered  in  the  discussion  of  scope 
requirements  for  TICARRS. 
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26.  Page  V-3,  Table  V-1,  c:  We  agree,  and  the  report  has  been  modified  to  reflect  this 
information. 

27.  Page  V-3  Table  V-1,  f:  This  capability  was  not  cemonstrated  to  IDA. 

28.  Page  V-3  Table  V-1,  g;  PQDRs  are  treated  in  the  text,  but  the  table  presents 
information  at  a  more  general  level.  The  REMIS  PMO  was  not  able  to  supply  IDA 
with  data  we  requested  that  would  have  demonstrated  serialized  part  maintenance 
history. 

29.  Page  V-4,  Table  V-1,  j:  The  report  has  been  modified  to  reflect  this  information. 

30.  Page  V-4,  Table  V-1, 1;  GCSAS  has  not  been  fielded  yet;  it  is  still  in  testing.  The 
text  of  the  report  has  been  revised  to  elaborate  on  this  point 

3 1 .  Page  V-5,  Para  a:  The  report  has  been  modified  to  reflect  this  information. 

32.  Page  V-5,  Para  A.3.a:  The  report  has  been  modified  to  reflect  this  information. 
The  IDA  team  observed  serious  problems  with  possession  change  information  at 
Langley  AFB. 

33.  Page  V-5  &  V-6,  Para  b:  ATERS  has  been  noted  in  the  report 

34.  Page  V-6,  Para  A.3.b:  The  report  has  been  modified  to  reflect  this  information. 

35.  Page  V-6,  Para  A.3.c:  The  report  has  been  modified  to  reflect  this  information. 

36.  Page  V-6,  Sentences  1  and  2:  The  report  has  been  modified  to  reflect  this 
information. 

37.  Page  V-6,  Para  2,  Last  sentence:  The  report  has  been  modified  to  reflect  this 
information. 

38.  Page  V-6,  Para  c:  The  report  has  been  modified  to  reflect  this  information. 
TICARRS’s  limitations  are  addressed  in  the  discussion  of  scope.  We  have  made 
some  changes  to  the  report  based  on  independent  IDA  findings. 

39.  Page  V-7,  Para  A.3.f:  Retrieval  of  narratives  was  not  demonstrated  to  IDA. 

40.  Page  V-9,  Para  A.3.g;  The  report  has  been  modified  to  reflect  this  information. 

4 1 .  Page  V-9,  Para  1 :  We  disagree.  A  review  of  screens  and  conversations  with  Plans 
and  Scheduling  personnel  at  the  squadron-level  at  Seymour  Johnson  AFB  confirm 
IDA  statements. 

42.  Page  V-9,  Para  3:  The  report  has  been  modified  to  reflect  this  information. 
Concerning  the  statement  about  B-1  contractor:  Rockwell  engineers  who  focus  on 
R&M  analyses  have  not  to  date  been  successful  in  using  REMIS.  They  rely  upon 
information  provided  to  them  from  site  representatives  at  the  B-1  bases.  However, 
the  report  has  been  slightly  modified. 

We  disagree  with  Litton’s  comment  about  contractor  needs.  Contractors  require 
historical  data  from  D056  tapes,  which  are  not  available  in  REMIS  (the  tapes  can 
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be  obtained  from  the  REMIS  office,  but  the  data  have  not  been  loaded  into 
REMIS).  REMIS  has  recent  data  only. 

43.  Page  V-12,  Para  2:  We  agree. 

44.  Page  V-13,  Para  2:  Block  number  tracking  in  REMIS  has  not  been  demonstrated. 
Our  impression,  based  on  discussions  with  users,  is  that  block  number  tracking  is 
used  for  more  than  just  TCTOs.  It  is  also  needed  to  track  R&M  status  by  block  and 
to  track  warranty  items. 

45.  Page  V- 14,  Para  p:  TICARRS  92  combines  TICARRS  87  and  SDS. 

46.  Page  V-16,  Para  B:  Scope  refers  to  breadth  and  depth  of  coverage,  not  to  the 
functions,  ease  of  use,  and  other  areas  of  concern  raised  here.  The  costs  of  the 
functions  mentioned  in  this  comment  have  been  estimated  and  are  presented  in 
Chapter  VII  of  the  report 

47.  Page  V-17,  Para  B,  Last  Subpara:  Given  the  low  usage  levels  of  REMIS  and  the 
results  of  our  own  observations,  we  believe  the  text  is  appropriate.  The  importance 
of  narratives  is  addressed  elsewhere  in  Chapter  V. 

48.  Page  V-21,  Para  1:  We  disagree. 

49.  Page  V-22,  Para  1 :  We  have  revised  the  text  to  make  our  sizing  consistent. 

50.  Page  V-22  and  V-23,  Para  2:  Noted.  We  stand  by  our  estimate. 

5 1 .  Page  V-23,  Table  V-3:  Noted.  We  stand  by  our  analysis. 

52.  Page  V-23,  Para  1 :  We  believe  that  we  are  correct  in  using  the  official  projections 
of  usage.  Utilization  of  a  fully-operational,  working  REMIS  system  should  be 
several  times  greater  than  existing  levels. 

53.  Page  V-25,  Para  2  and  Figure  V-3:  Noted. 

54.  Page  V-29,  Para  c:  We  agree  and  have  modified  the  report  to  reflect  this 
information. 

55.  Page  V-32,  Para  C.l.c.(3'  2nd  Subpara:  Noted. 

56.  Page  V-32,  Para  4:  The  report  has  been  modified  to  reflect  this  information. 

57.  Page  V-33,  Para  C.l.d:  Noted. 

58.  Page  V-36,  Para  C.l.f:  Noted.  We  stand  by  our  hardware  sizing  estimate. 

59.  Page  V-36,  Para  F,  2nd  Subpara:  Noted.  We  stand  by  our  hardware  sizing 
estimate,  and  we  have  included  additional  detail  in  the  report  justifying  it 

60.  Page  V-36,  Para  C.l.f,  3rd  Subpara:  Noted. 

6 1 .  Page  V-37,  Para  1 :  We  have  modified  our  discussion  of  this  issue  to  clarify  it. 

62.  Page  V-37,  Para  2:  Noted. 
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63.  Page  V-38,  Para  A;  See  our  responses  to  the  referenced  comments  (notes  1-3,  14, 
and  15). 

64.  Page  V-38,  Para  a,  “REMISTALK  has  poor  training”:  While  training  may  have 
received  good  ratings  in  the  course  critiques,  opinions  of  the  effectiveness  of 
training  may  change  weeks  after  the  class,  as  trainees  try  to  implement  what  they 
have  learned  in  the  field. 

65.  Page  V-38,  Para  C.2.a:  It  is  true  that  only  ACC  was  surveyed  in  our  study. 
However,  USAFE  and  PACAF  are  heavy  TICARRS  users.  The  text  has  been 
altered  to  address  this  comment. 

66.  Page  V-39,  Para  A:  We  have  not  discounted  student  ratings  on  REMIS  training. 
Our  information  is  that  contractors  are  having  trouble  accessing  and  using  REMIS, 
and  they  have  told  us  that  they  thought  training  was  not  adequate.  Training  goes 
beyond  formal  classroom  training  and  includes  on-line  support  during  normal  use 
periods. 

67.  Page  V-39:  The  IDA  study  team  found  no  contractor — Rockwell,  Lockheed 
Georgia,  Lockheed-Fort  Worth,  or  McDonnell-Douglas — who  preferred  using  on¬ 
line  REMIS  to  receiving  D056  tapes. 

68.  Page  V-45,  Para  D:  The  report  has  been  modified  to  include  this  comment. 
However,  our  intent  was  to  note  the  preference  of  TICARRS  over  REMIS  by 
those  who  had  used  or  were  familiar  with  both  systems. 

69.  Page  V-46,  Para  D.  1 :  This  potential  capability  may  improve  the  situation,  but  it  has 
not  been  demonstrated. 

70.  Page  V-47,  Para  2:  We  agree  and  have  modified  the  report  to  reflect  this 
information. 

7 1 .  Page  V-52:  We  agree.  CAMS  has  proposed  a  similar  system. 

72.  Page  V-52,  2nd  Subpara  and  Figure  V-8;  Noted. 

73.  Page  V-54,  Para  D.3:  We  have  revised  the  security  discussion  significantly. 

74.  Page  V-64,  Para  E.2:  We  have  revised  the  discussion  to  clarify  the  specific 
advantages  and  disadvantages  of  the  current  systems  in  migrating  to  the  future. 

75.  Page  VII-3,  First  Paragraph:  We  have  revised  this  entire  analysis  based  on 
additional  information  provided  by  the  Standard  Systems  Center. 

76.  Page  VII-4,  Table  VII- 1:  We  agree  and  have  modified  the  report  to  reflect  this 
information. 

77.  Page  VIM  and  VII-32:  The  timing  of  regional  processing  center  costs  has  been 
addressed  with  our  new  method  of  analysis.  On  the  issue  of  modifications,  we 
apree,  and  we  have  assumed  that  CSRDs  would  continue.  The  cost  for  base 
representatives  has  been  addressed  with  our  new  method. 
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78.  Page  VIl-4  and  VIl-6,  System  Integration  Test;  The  costs  are  different  because 
CAMS  has  an  additional  integration  to  perform — CAMS  with  other  SBLC 
applications — and  TICARRS  does  not  use  the  SBLC. 

79.  Page  VII-5,  Approved  Functional  Modifications:  The  costs  represent  the  effort  to 
finish  the  backlog  of  approved  CSRDs. 

80.  Page  VII-6:  We  agree  and  have  modified  the  report  to  reflect  this  information. 

8 1 .  Page  VII-6  (CAMS  Extensions  and  New  Weapon  Support):  We  agree  and  have 
modified  the  report  to  reflect  this  information. 

82.  Page  VII-7,  Para  2E;  We  agree  and  have  modified  the  report  to  reflect  this 
information. 

83.  Page  Vn-10,  Table  Vn-2,  1.2,  and  VII-1 1,  Para  4.b:  We  disagree  and  have  made 
independent  estimates. 

84.  Page  VII-13,  Para  B  and  C;  We  agree  and  have  modified  the  report  to  reflect  this 
information. 

85.  Page  VO- 16:  We  agree  with  the  comment  on  the  TICARRS  MAISRC  requironent. 
We  assumed  8  months  for  TICARRS  functional  enhancements.  The  comment  may 
have  been  prompted  by  confusion  of  what  we  mean  by  scope.  In  the  report,  scope 
is  the  number  of  weapon  systems  supported.  We  have  allowed  29  months  for 
expanding  scope  and  loading  data. 

86.  Page  Vfl- 17,  Table  Vn-4: 

Adjustment  to  TICARRS  Non-recurring  Hardware  Costs — We  agree  and  have 
modified  the  report  to  reflect  this  information. 

Hardware  Purchases — ^We  assumed  that  TICARRS  would  have  to  purchase  new 
equipment  to  completely  support  expansion. 

Training:  We  believe  that  trainee  costs  should  be  excluded,  just  as  the  cost  of 
people  using  the  system  are.  In  any  event,  we  believe  the  Litton  estimates  are  too 
high,  since  each  user  is  trained  for  2  to  4  hours,  not  three  weeks.  We  have  not 
included  student  time  in  the  estimates  for  any  system. 

Adjustment  to  Hardware  and  Software  Maintenance  Costs — ^We  stand  by  our 
estimate  of  the  number  of  computers  required  for  TICARRS. 

Civilian  and  Military  Personnel — We  agree  and  have  modified  the  report  to  reflect 
this  information. 

87.  Page  VII- 18  and  VII- 19,  Mainframe  sizing  for  TICARRS:  We  disagree.  The 
CAMS  tiimsactions  cited  were  for  a  single  shift,  versus  a  full  day  of  TICARRS 
transactions.  The  assumptions  underlying  the  Litton  analysis  are  incorrect. 

8  8 .  Page  VII-2 1 :  We  agree,  and  we  have  included  these  costs  in  our  estimate. 

89.  Page  Vn-21,  Table  Vn-5; 
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Communications-electronics — These  enhancements  have  been  included  under  data 
base  initialization  for  $1.7  million. 

Supply  System — We  disagree.  We  have  included  the  cost  of  the  SBLC  interface. 

FDWWs  and  DIREPs  identified  at  the  Seymour  Johnson  Operational 
Assessment — These  have  been  included  in  our  costs. 

Costs  for  loading  REMIS  data  into  TICARRS — We  agree  and  have  modified  the 
report  to  reflect  this  information. 

TICARRS  Hardware  Costs — We  stand  by  our  estimate. 

Initialization  Costs  for  Trainers  and  ATE  Equipment — We  agree  and  have  modified 
the  report  to  reflect  this  information. 

Lines  of  Code — We  disagree,  and  we  have  included  additional  detail  on  lines  of 
code  in  the  final  report. 

90.  System  Software  Interfaces:  Our  TICARRS  cost  estimate  includes  0.75  staff-years 
per  interface,  even  more  than  the  Litton  estimate. 

9 1 .  Page  VII-23,  Para  D:  Our  estimate  of  10  percent  of  application  software  includes 
user  documentation  only.  We  have  revised  the  report  to  include  costs  of  applying 
the  DoD  standards. 

92.  Page  VII-24:  We  agree  and  have  modified  the  report  to  reflect  this  information. 

93.  Page  VII-24,  Table  VII-6:  We  agree  and  have  added  these  systems  to  our  list 

94.  Page  VII-27,  Para  3:  We  disagree  and  have  not  considered  trainee  cost  for  any 
system. 

95.  Comment  on  additional  alternatives:  Our  task  was  to  do  a  cost  and  operational 
comparison  of  CAMS/REMIS  and  TICARRS,  and  we  have  analyzed  only  the 
alternatives  necessary  to  do  this. 
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PAGE  I‘l  AMD  1-2 


The  report  specifies  that  the  GAO  found  serious  problems  with 
REMIS  software.  Do  the  difficulties  affect  the  overall 
capedjility  of  the  REMIS  to  support  the  over  two- thousand  major 
end  items?  These  comments  may  refer  to  the  "data  accuracy" 
issue  that  has  been  raised  in  the  past.  Please  see  Attachment 
1,  T.  King  letter  to  Mr.  Dave  Roberts,  dated  February  3,  1993 
on  this  sxibject. 

No  reference  to  the  recent  Operational  Assessment  of  REMIS 
subsystems  (EIMSURS  euid  PPS)  has  been  included.  Aren't 
Operational  Assessment  findings  appropriate  for  the  study 
effort?  We  think  so. 

Several  base  level  studies  of  CAMS  are  referenced,  but  no 
specific  identification  of  these  studies  is  provided. 

PAGE  II -1  PARA  A.l 

The  report  states  that  the  major  functions  of  CAMS  are  listed, 
but  a  large  number  of  the  major  functions  are  missing, 
including;  Utilization  Reporting,  C-E  Subsystem,  Trainer 
Subsystem,  ATE  Subsystem,  Configuration  Mamagement,  etc. 

The  report  goes  into  great  detail  as  to  what  functions  TICARRS 
provides,  for  example.  Page  II -10  2nd  paragraph  lists:  cdjort 
analysis,  aircraft  status  and  utilization  measurement  and 
analysis,  flight  scheduling,  pilot  debriefing,  etc...  While 
these  same  fiinctions  are  provided  in  CAMS/REMIS,  little,  or  no 
mention  is  made  of  that  fact. 

Litton  would  be  happy  to  provide  a  conplete  listing  of 
functions  provided  by  the  CAMS/REMIS  system. 

For  a  listing  of  functions  that  TICARRS  does  not  provide  see 
Attachment  2 . 

PAGE  II-l  PARA  A.l  PAGE  II-6  PARA  B.l  PAGE  II-9  PARA  C.l 

The  system  descriptions  contained  in  the  referenced  paragraphs 
do  not  compare  the  systems  at  the  same  level .  The 
introductory  sentences  are  not  of  a  similar  nature.  CAMS  is 
introduced  by  the  phrase:  "...major  functions  provided". 
REMIS  is  introduced  by  the  phrase:  "...these  siibsystems  are 
briefly  described",  and  TICARRS  is  introduced  by  the  phrase: 
"... Its  key  features  are . . . " . 

For  a  valid  conparison  to  occur,  the  systems  must  be  described 
in  like  manner,  by  function,  by  subsystem  or  by  key  feature. 
Most  of  the  "key  features"  listed  for  TICARRS  are  also 
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capcQsilities  of  CAMS/REMIS,  but  the  report  does  not  point  this 
out. 


The  description  of  TICARRS  lists  organizations  with  access  to 
the  system,  but  no  list  is  provided  for  CAMS/REMIS.  The 
CAMS/REMIS  list  is  more  extensive  and  also  includes  other 
automated  data  systems.  Attachment  3  shows  the  2516  REMIS 
users  as  of  July  1,  1993.  All  26  organizations  shown  on  this 
chart  presently  have  access  to  REMIS,  while  only  6  of  them 
have  access  to  TICARRS: 


ACCESS  TO  SYSTEM 


ORGANIZATION 

CAMS/REMIS 

TICARRS 

AFMC 

YES 

YES* 

AECC 

YES 

NO 

AFSOC 

YES 

NO 

PACAF 

YES 

YES 

ATC 

YES 

NO 

AMC 

YES 

NO 

AFIC 

YES 

NO 

CONTR 

YES 

YES 

AF  RESERVES 

YES 

YES 

ANG 

YES 

YES 

ACC 

YES 

YES 

USAF 

YES 

YES 

SPACECOM 

YES 

NO 

USAFE 

YES 

YES 

AF  ACADEMY 

YES 

NO 

AF  SAFETY  AGENCY 

YES 

NO 

AFIT 

YES 

NO 

CENTRAL  COMMAND 

YES 

NO 

AF  COST  ANAL  AGENCY 

YES 

NO 

AFOTEC 

YES 

NO 

NAT  WEATHER  SERVICE 

YES 

NO 

AUDIT  AGENCY 

YES 

NO 

ACCOUNTING  &  FINANCE 

YES 

NO 

AF  HISTORICAL  RES  CTR 

YES 

NO 

NAVY 

YES 

NO 

OSD 

YES 

NO 

DLA 

YES 

NO 

SAF 

YES 

NO 

♦ONLY  OGDEN  ALC  HAS  ACCESS 
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PAGE  ZZ-2  BULLET  3 


This  statement  seems  to  that  CAMS  only  updates  CISIS  and 

is  misleading.  The  system  provides  single  terminal  input  to 
the  CAMS  data  base  auid  also  interfaces  with  CEMS  (D042)  via  a 
daily  tape.  The  CAMS  users  are  able  to  control  inventory, 
status,  utilization,  and  configuration  of  all  engines  and 
consonants  at  their  home  station,  as  well  as  when  they  are 
deployed. 

PAGE  ZZ-3  PARA  2  2HD  SUB  PARA 

The  reference  to  the  size  of  CAMS  specifies:  ”1.1  million 
unique  Source  Lines  of  Code  (SLOC) . ”  The  reference  to  the 
size  of  TICARRS  specifies:  ”The  application  software  is  in 
COBOL  74  (approximately  850  thousand  lines  of  code)."  No 
software  sizing  data  for  REMIS  is  provided.  This  does  not 
appear  to  be  a  valid  conparison. 

PAGE  ZZ-2  and  ZZ-3 

The  following  should  be  included  as  additional  functions  added 
to  CAMS  over  the  period  1985-1992: 

1)  Automatic  Test  Equipment  Reporting  System  (ATERS) .  This 
sxibsystem  provides  on-line  access  to  a  data  base 
containing  organization,  equipment,  status,  eind 
utilization  data  for  assigned  test  equipment.  It  also 
keeps  track  of  the  capability  of  various  test  stations  to 
test  particular  parts.  This  permits  the  Avionics 
Intermediate  Shop  (AIS)  to  update  the  appropriate  data  in 
the  data  base  in  cui  on-line  mode  as  events  occur  which 
affect  assigned  test  equipment. 

2)  Product  Quality  Deficiency  Reporting  (PQDR)  Subsystem. 

The  PQDR  Subsystem  reports  known  or  suspected 

deficiencies  for  equipment,  weapon  systems,  or  related 
coitponents  and  records  exhibit  disposition  instructions 
and  data.  This  subsystem  has  an  interface  to  INFOCEN 
that  allows  approved  PQDRs  to  be  transmitted  directly  to 
INFOCEN  via  CAMS. 

PAGE  ZZ-4  PARA  A.3.b  PAGE  ZZ-8  PARA  B.3.b,  and  PAGE  ZZ-12  PARA 

C.3.b 
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The  "Perceived  Problems"  descriptions  contained  in  the 
referenced  paragraphs  do  not  describe  the  syst^  problems  at 
the  same  level.  The  introductory  sentences  are  not  of  a 
similar  nature.  CAMS  is  introduced  as:  "CAMS  has  been 
accused  by  some  of  its  users  of  a  wide  variety  of 
deficiencies.",  REMIS  is  introduced  as:  "Several  issues  that 
affect  the  satisfactory  operation  of  REMIS  are:  ",  and  TICARRS 
is  introduced  as:  "A  number  of  existing  Air  Force  functions 
are  not  provided  in  the  current  version  of  TICARRS . " 

For  a  valid  con^arison,  the  perceived  problems  should  be 
described  in  like  manner,  by  user- voiced  deficiencies, 
operational  issues,  or  by  missing  Air  Force  functions.  It  is 
not  clear  whether  or  not  the  "deficiencies"  in  CAMS  are  in  the 
areas  where  TICARRS  is  totally  missing  the  function.  If  so, 
there  is  no  reason  to  assume  that  a  TICARRS  inplementation  of 
the  function  would  not  have  the  same  deficiency.  No  mention 
of  operational  considerations  for  TICARRS  is  included.  Since 
so  many  critical  functions  are  missing  from  TICARRS,  the 
development  effort  required  to  add  them  will  almost  certainly 
cause  operational  concerns  similar  to  those  perceived  for 
REMIS,  especially  in  the  area  of  performauice. 


PAGE  II -5  PARA  3 

There  appears  to  be  confusion  with  regard  to  the  roles  of  the 
Data  Base  Administrator  (DBA)  and  the  Data  Base  Manager  (DBM)  . 
The  functions  of  the  DBA  and  the  DBM  are  decidedly  different 
but  the  report  doesn't  appear  to  recognize  the  differences. 


PAGE  II -6  PARA  B.l  1ST  SXJBPARA 

First  sentence  states  that  REMIS:  ".  .  .is  meant  to  serve  as 
a  centralized  source  of  fleet -wide  weapon  system  configuration 
and  historical  maintenance  data  for  all  Air  Force  weapon 
systems  and  support  equipment." 

REMIS  does  serve  as  a  centralized  data  source;  this  is  not  a 
future  capability  (as  inferred  by  the  phrase:  "...  meant  to 
be  .  .  .". 

Fleet -wide  weapon  system  configuration  and  historical 
maintenance  data  are  only  two  of  the  major  functions  of  REMIS. 
The  list  should  include  Inventory  Control  through  maintaining 
the  Air  Force  Master  Inventory,  Inventory  Assignment,  and 
Inventory  Possession:  weapon  system  status,  both  current  and 
historical,  for  aircraft,  missiles,  trainers,  automated  test 
equipment,  communication  -  electronics  equipment,  and 
utilization  information  such  as  actual  euid  allocated  flying 
hours,  actual  sorties,  landings,  and  full  stop  landings. 
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PAGE  ZI*6  PASA  B.l  3SD  SUBPASA 

The  list  of  functions  REMIS  is  to  provide,  "when  fully 
operational",  has  been  provided  and  is  currently  available. 
The  functions  listed  provide  the  framework  that  specific 
application  software  must  function  within.  Some  of  the 
functions  of  a  similar  nature  which  have  not  been,  and  should 
be ,  included  are : 

Open  communication  support  architecture 

Data  conqpatibility  edits 

Data  con^leteness  edits 

Data  reconciliation 

Air  Force  steuidard  algorithms 

1  year  of  detail  data  and  5  years  of  summary  data 
on-line 

Archival  and  Retrieval  of  detail  data 
Integrated  report  capabilities  across  weapon 
. systems 

Data  types  and  geographic  location  for  user 
specified  subject,  time  spem,  and  other  relevant 
criteria  for  both  detail  euid  summary  data 
World-wide  Electronic  Mail 

Controlled  user  access  by  read,  add,  change  and 
delete 

On-line  edits  as  a  feedback  mechanism  to  people 
entering  data  in  order  to  facilitate  detailed  and 
accurate  reporting.  Up  to  10  error  conditions  may 
be  identified  for  each  transaction  attempt. 

On-line  help  screens  for  all  applications  as  well 

as  on-line  system  level  help 

System  to  system  interfaces 

Inbound  and  outbound  data  tremsfers 

Etc. 

PAGE  II -7,  PARA  B1 

The  EIMSURS  subsystem  also  provides;  data  reconciliation;  many 
Master  Validation  taUales  such  as  Work  Unit  Code;  Command  Code; 
Allowable  Condition  Status  Code;  65-110  Organization  Number; 
Kind  and  Type;  Type  Utilization  Code/Mission  Symbol,  etc.; 
pushdown  of  the  centrally  maintained  edit  and  validation 
tables  to  CAMS  bases  worldwide;  input  interfaces  with  systems 
such  as  AICARRS  (G081)  for  inventory,  status,  and  utilization 
and  Air  Staff  {K002)  for  allocated  flying  hours;  output 
interfaces  with  systems  such  as  D087F,  D160,  D200,  G086A, 
G075,  K008,  K002  and  contractors  to  provide  inventory,  status, 
and  utilization  information  in  a  format  customized  to  meet  the 
needs  of  the  receiving  system. 

The  EIMSURS  subsystem  also  provides  information  to  the  other 
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REMIS  subsystems  since  flying  hours  and  sorties  are  needed  to 
calculate  many  of  the  R&M  predictors  such  as  Meeui  Time  Between 
Failure  (MTBF)  or  Meaui  Time  Between  Maintenance  Mauihour 
(MTBMMH)  . 

REMIS  EIMSURS  receives  hourly  updates  from  base  level  users 
for  inventory  possession,  status,  and  utilization;  EIMSURS 
also  provides  on-line  entry  of  this  data.  Both  current  and 
historical  status  information  is  available  fleet -wide  in 
REMIS.  Authorized  users  may  ask  REMIS  to  provide  information 
such  as  the  current  Air  Force  wide  Mission  Capadjility  Rate  for 
all  F-15  aircraft  or  for  amy  combination  of  MDS,  organization, 
location,  purpose  code,  commauid  code,  etc. 

EIMSURS  is  the  only  automated  system  which  supports  full 
aircraft  assignment  functions  at  both  the  comnand  and 
organizational  levels. 

A  majority  of  these  functions  are  not  in  the  current  TICARRS, 
and  costs  associated  with  enhamcing  TICARRS  to  include  these 
functions  are  not  addressed  in  Paragraph  VII.C. 

PAGE  II -7,  PARA.  B1 

The  description  of  the  PPS  subsystem  is  not  complete  when 
compared  to  the  description  of  the  saune  function  in  TICARRS. 
Some  of  the  missing  functionality  includes: 

Discrepancy  and  Corrective  Action  Narratives 
Data  compatibility  and  completeness  edits 
Bad  actor  identification 

Reporting  of  repeat  or  recurring  problems  with 
particular  Work  Unit  Codes 
Cannot  duplicate/re- test  OK  auialysis 
Serial  number  tracking 

Fault  isolation/analysis/resolution  by  providing 
failure  typing  (inherent,  induced,  no  defect) 

Not -RepairaUDle- This -Station  (NRTS)  analysis 


PAGE  II- 8  AMD  II-9  PARA  3.B  AMD  3.C 

LCS  has  a  system  performance  team  in  place  to  analyze  and 
address  all  issues  pertaining  to  system  performance.  This 
team  is  developing  plans  and  schedules  to  repartition  data 
stored  in  the  REMIS  data  base  tables,  reallocation  of  disk 
space,  and  implementation  of  the  Operating  System's  parallel 
processing  capcdDility  to  its  fullest  extent.  This  teeun  has 
also  developed  many  recommendations  for  changes  in  application 
software  to  improve  processing  speed.  It  is  anticipated  that 
these  changes  will  be  implemented  by  the  PMO  as  resources 
become  availcd)le.  Also  under  consideration  are  plans  for  a 
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partial  reconfiguration  of  \HQ1  and  \H02  hardware  and  movement 
of  REMISTALK  from  \HQ1  to  \HQ2. 

The  scope  of  REMISTALK  is  regularly  increased  to  meet 
additional  user  requirements.  This  is  being  accon5)lished  via 
creation  of  additional  Reportedsle  Data  Base  Areas  (RDAs)  or 
the  addition  of  taibles  to  the  current  RDAs.  This  effort  does 
not  require  software  enhancement,  just  the  description  of  data 
elements  auad  tables  and  loading  of  information  concerning 
these  RDAs  to  the  REMIS  data  base. 

The  implementation  of  a  REMIS  information  center  data  base 
utilizing  the  current  development  hardware  (Tandem  CLX)  at  FOC 
will  resolve  the  scope  and  response  time  issue  for  ad  hoc  and 
standard  reports.  Movement  of  these  activities  to  the  CLX 
will  free-up  resources  on  \HQ1  and  \HQ2  to  meet  additional 
user  requirements  auid  greatly  enhance  system  performance. 


PAGE  II -9  SECOND  BULLET  TOP  OF  PAGE 

The  report  states  that,  "Contractors  have  had  difficulty 
getting  access  to  REMIS  due  to  imposed  security  steps".  This 
statement  is  incorrect.  The  Air  Force  policy  for  security  has 
not  yet  been  defined  cuad,  until  it  is,  the  Contractors  are 
expected  to  conplain  about  any  system  whether  it  is 
CAMS/REMIS,  TICARRS,  or  any  other. 


PAGE  II -9  PASA  Cl 

This  paragraph  is  in  error  and  conflicts  with  other  portions 
of  the  report.  It  states  that  TICARRS  provides  a  "deployedale 
system" .  Other  portions  of  the  study  clearly  state  that 
TICARRS  does  not  provide  a  deployable  system.  See  Pages  V-4, 
Tcible  V-1,  and  Page  V-15,  "neither  CAMS  nor  TICARRS  is  able  to 
support  such  deployments  today" . 

Most  significantly,  costs  were  added  to  CAMS  to  develop  this 
capability  (Page  VII- 6,  Paragraph  lb,  CAMS  Extensions  and  New 
Weapon  Support)  in  CAMS.  However,  similar  costs  were  not 
added  to  the  TICARRS  cost  estimates  (see  Pages  VII -19  and  20, 
Paragraph  2b.)  Since  neither  system  has  the  deployable 
capability,  the  costs  to  develop  it  should  be  addet  to  both 
CAMS  and  TICARRS  to  portray  valid  cost  coit^arisons.  In 
addition,  no  hardware  costs  were  included  for  a  deployable 
system.  Both  new  software  and  hardware  will  be  needed. 

The  Air  Force  has,  on  a  number  of  occasions,  evaluated  Major 
Command  and  agency  requireme  ts  for  a  deployaUsle  maintenance 
data  collection  system.  T  'jse  reviews  have  taken  place  at 
least  each  year  since  1984,  and  to  date,  no  clear  cut 
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requirement  has  been  estoiblished  for  a  deployable  system. 

The  latest  review  process  is  underway  at  this  time.  Air 
Combat  Command  (ACC)  is  now  taking  the  lead  in  accumulating 
all  requirements  of  a  deployable  CAMS  system,  in  order  to 
estcdjlish  a  firm  requirement  which  will  be  presented  to  the 
CAMS/REMIS  Functional  Review  Board  members  at  their  next 
meeting.  ACC  then  intends  to  modify  the  current  Functional 
Description  and  furnish  the  document  to  Hq  USAF  for 
disposition.  The  point  of  this  discussion  is  that  the  Air 
Force  has  not  yet  estaiblished  a  firm  requirement  for  a 
deploycUole  CAMS  system;  therefore,  the  statement  made  by  the 
IDA  evaluation  team  appears  to  lack  a  proper  foundation. 
Until  the  requirements  are  knovm,  no  systems  should  be 
recommended  or  selected  over  another. 

Litton  Con^juter  Services  (LCS) ,  among  others,  has  recommended 
a  development  concept  for  a  Deployadsle  CAMS  (DCAMS)  system. 
Litton' s  concept  takes  advantage  of  existing  CAMS  code, 
resident  on  the  UNISYS  1100,  by  trainsporting  the  COBOL  code 
from  the  mainframe  platform  to  a  personal  con?)uter  (PC) 
environment.  At  project  completion,  the  system  would  contain 
a  relational  data  base,  using  open  systems  operations.  The 
key  to  the  LCS  concept  is,  the  end  user  view  of  input  screens 
and  output  reports  remains  exactly  the  same  in  the  PC  version 
as  in  the  mainframe  version.  Thus,  no  end  user  training  is 
required  to  operate  the  PC  version  of  CAMS,  since  the  code  on 
the  PC  version  operates  exactly  as  the  mainframe  would.  This 
is  extremely  inportant  to  both  management  and  the  users,  since 
no  change  in  operations  is  required  when  a  unit  deploys.  The 
user  should  not  be  required  to  operate  two  different  systems 
(one  at  home  station  and  another  at  a  deployed  site)  .  LCS  has 
made  its  views  )cnown  to  many  key  decision  makers  in  the  Air 
Force,  and  our  concept  has  been  generally  accepted. 

The  Dynamics  Research  Corporation  (DRC)  model  of  a  deployedDle 
system,  called  Squadron  Centered  Logistics  System  (SCLS) ,  has 
vastly  different  input  and  output  views  from  that  of  CAMS. 
Most  units  deploy  on  a  limited  basis  (2-3  times  per  year)  . 
Providing  the  end  user  a  different  "deployed"  system,  one 
which  is  different  from  the  norm  (CAMS) ,  will  probably  result 
in  confusion  and  set  the  stage  for  errors  in  data  collection, 
and  the  loss  of  data.  In  turn,  when  returning  from  the 
deployed  site,  confusion,  errors,  and  data  loss  may  occur 
until  additional  retraining  takes  place. 

The  fact  that  Air  Force  requirements  for  a  deployable  systems 
have  not  yet  been  firmly  established,  and  the  observation  that 
TICARRS  would  be  suitable  for  a  deployable  system  are  in 
conflict  with  each  other. 
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PAGE  II-IO  PASA  C.l 


References  to  "RAM  for  the  F-117A,  and  INS"  do  not  seem 
appropriate.  These  are  not  currently  TICARRS  supported 
functions . 

PAGE  ZI-10  PARA  3  LAST  SENTENCE 

Warranty  tracking  is  availeOale  for  F-15  by  query  only. 
TICARRS  provides  no  notification  of  warranty  parameters.  The 
F-16  users  must  query  the  data  base  by  inputting  part  numbers 
the  user  knows  are  under  warranty. 

PAGE  II-ll  PARA  C.2 

Reference  to  SDS  and  F-117A  are  not  appropriate  since  SDS  is 
not  a  part  of  TICARRS  and  the  F-ll7A's  are  now  fully  supported 
by  CAMS  and  have  been  since  they  became  unclassified. 

PAGE  11-12  PARA  B 

The  problems  cited  in  the  TICARRS  92  test  are  not  a  matter  of 
perception  but  real  problems.  Please  refer  to  General 
Hammond's  letter  to  Senator  Inouye,  dated  May  24,  1992, 
Attachment  4 . 

The  statement,  "Many  aircraft  types  supported  by  CAMS/REMIS 
are  not  now  being  supported  by  TICARRS"  is  misleading.  Please 
see  Attachment  1  which  shows  the  items  supported  by  CAMS/REMIS 
and  TICARRS: 


ITEM 

CAMS/REMIS 

TICARRS 

• 

Aircraft 

10,555 

2,800 

• 

Missiles 

4,395 

None 

• 

Communications 

Electronics 

1,054,000 

None 

• 

Aerospace  Ground  60,000 

Equipment  (including 

Automated  Test  Equipment) 

500 

• 

Simulators  &  Trainers 

1,081 

None 

PAGE  11-12  PARA  C 

DRC's  analysis  of  what  work  would  need  to  be  performed  to 
upgrade  TICARRS  does  not  seem  to  be  the  best,  unbiased  source 
for  this  information. 

PAGE  11-12  PARA  II.C.3.b 
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Describing  TICARRS  "perceived  problems"  in  the  terms  of  Air 
Force  functions  does  not  address  shortcomings  within  the 
limited  functions  that  do  exist.  Some  shortcomings  that 
should  be  addressed  include: 

Data  completeness:  no  interface  exists  for  TICARRS  to 
receive  information  on  PMEL;  the  ATE  functions  within 
TICARRS  do  not  address  all  ATE  equipment;  failure  data  on 
parts  utilized  in  more  than  the  F016  and  F015E  weapon 
systems  ia  incomplete  due  to  the  lack  of  visibility  of 
the  other  weapon  systems,  etc. 

Data  accuracy:  TICARRS  does  not  have  am  interface  with 
D043  and,  therefore,  does  not  have  an  Air  Force  approved 
part  number  master;  this  permits  inaccurate  data  to  enter 
the  system. 

Operational  considerations,  such  as  the  frequency  and 
number  of  PLP  (ad  hoc)  report  requests  are  not  captured. 
This  prevents  the  staindardization  amd  optimization  of  ad 
hoc  reports;  base  level  users  are  required  to  perform 
dual  inputs  since  TICARRS  cannot  meet  all  of  the  users 
basic  needs. 

Communication  to  provide  users  access  to  TICARRS  is  based 
upon  dedicated  leased  lines.  If  the  TICARRS  user 
community  was  expanded  to  be  equivalent  to  that  of 
CAMS/REMIS,  the  number  of  lines  would  also  have  to  be 
expanded.  CAMS/REMIS  en:?)loys  a  much  more  cost  effective 
methodology  to  achieve  user  connectivity. 

TICARRS  is  based  upon  a  Network  data  base  structure. 
This  type  of  data  base  does  not  support  accurate  modeling 
of  the  con^lex  relationships  that  exist  between  R&M 
information.  REMIS  is  based  upon  a  relational  data  base 
structure  and  more  accurately  represents  R«M  information. 
Maintenance  costs  for  major  upgrades  to  Network  data  base 
systems  are  generally  higher  than  those  for  relational. 

See  also  Attachment  2. 


Major  functions  provided  by  CAMS/REMIS  but  not  addressed 
include : 

Allocated  flying  hours. 

Current  status  for  Aircraft,  Missiles,  Trainers, 
and  Communication- Electronics . 

Inventory  Assignment  for  Aircraft  at  both  the 
command  and  orgeuiizational  level  including 
information  such  as  assignment  directive,  and 
geographic  location. 
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Support  for  aircraft  other  than  F-16  and  F-15E. 
This  is  major  since  most  other  aircraft  do  not 
manage  by  block  number,  some  use  configuration 
identifiers. 

User  Maintainadsle  Validation  tcibles  such  as 
Standard  Reporting  Designators  (SRD) . 

Air  Force  standard  algorithms.  Those  utilized  by 
the  F016  and  F015E  community  are  not  necessarily 
acceptable  to  the  Air  Force  R&M  community  as  a 
whole . 

Multiple  Status  for  Communication- Electronics 
equipment . 

Maintenance  of  TCTO  Master  information. 

The  majority  of  these  functions  do  not  exist  in  the  current 
TICARRS  and  are  not  addressed  in  either  Paragraph  VI. C,  Areas 
of  Enhancement  for  TICARRS,  or  Paragraph  VII. C,  Cost  Estimates 
for  TICARRS. 

PAGE  IV-2  PARA  IV. B. 2 

Shouldn't  the  list  of  activities  include  the  F-117A  program 
office?  F-117A  personnel  have  had  the  opportunity  to  utilize 
SDS  and  to  utilize  CAMS/REMIS.  The  Air  Force  chose  CAMS/REMIS 
for  the  F-117A. 

PAGE  V-3  TABLE  V-1 

Inventory  Possession  and  Inventory  Assignment  should  not  be 
lumped  together;  they  are  two  separate  functions.  REMIS  does 
receive  narrative  data  on  debriefing  failure  history  and 
corrective  actions  under  DDN. 

REMIS  does  contain  serialized  part  maintenance  history  and 
provide  bad  actor  identification  by  WUC. 

REMIS,  under  PPS,  tracks  and  outputs  all  cannibalization 
actions . 

TCTO  status  has  been  received  from  CAMS  since  PPS 
implementation. 

PAGE  V-3  TABLE  V-1, a 

Equipment  Inventory  -  Assignment  and  Possession  are 
separate  and  discreet  functions  and  should  not  be 
combined.  REMIS  supports  full  assignment  capability  for 
command  and  organizational  levels  and  includes  both 
current  and  historical  information  for  all  aircraft. 
TICARRS  does  not  support  this  function.  The  costs 
associated  with  adding  this  function  to  TICARRS  are  not 
included  in  Paragraph  VII. C. 
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REMIS  maintains  both  current  and  historical  inventory 
possession  information  for  aircraft,  missiles,  trainers, 
automated  test  equipment,  emd  communication-electronics 
equipment.  TICARRS  does  not. 

REMIS  maintains  the  Air  Force  Master  Inventory  including 
information  such  as  acceptance  date,  lot  number,  block 
number,  configuration  identifier  for  all  aircraft  types, 
missiles,  trainers,  AGE,  ATE,  C-E,  PMEL,  Munitions,  and 
Parts.  TICARRS  does  not. 

PAGE  V-3  TABLE  V-l,b 

CAMS/REMIS  supports  Multiple  Status  for  C-E  and  all  ATE 
equipment.  TICARRS  does  not. 

PAGE  V-3  TABLE  V-l,c 

REMIS  provides  the  capability  to  con^are  actual  flying  hours 
to  Air  Staff  allocated  flying  hours.  TICARRS  does  not  have 
allocated  flying  hours. 

REMIS  supports  Air  Staff  by  providing  actual  flying  hours, 
sortie,  and  leuiding  data  by  Mission  Symbol  by  Program  Element 
code  for  each  possessed  aircraft.  TICARRS  does  not  have 
utilization  data  by  Program  Element  Code  (an  Air  Staff 
requirement)  and  does  not  support  an  interface  with  Air  Staff. 

PAGE  V-3  TABLE  V-l,£ 

The  ted)le  is  incorrect.  REMIS  does  maintain  debriefing 
discrepancy  narratives. 

PAGE  V-3  TABLE  V-l,g 

An  important  aspect  of  Maintenance  Scheduling/Reporting  is 
Product  Quality  Deficiency  Reports  (PQDRs) .  This  function  is 
missing  from  the  ted)le  and  should  be  included.  CAMS/REMIS 
supports  this  function,  TICARRS  does  not. 

REMIS  supports;  job  tracking.  Failure  Histories,  including 
narratives.  Serialized  Part  Maintenance  History,  and  "Bad 
Actor"  Identification.  The  tcdDle  should  be  corrected. 

PAGE  V-4  TABLE  V-l,j 

REMIS  does  support  cannibalization  tracking. 

PAGE  V-4  TABLE  V-1,1 

TCTO  Management  consists  of  two  major  functions;  TCTO  Master 
information  and  TCTO  status.  The  REMIS  GCSAS  supports  both  of 
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these  functions.  TICARRS  does  not;  only  TCTO  status  is 
supported  by  TICARRS. 

PAGE  V-5  PARA  a 

CAMS  only  provides  commeind  of  assignment.  TICARRS  does  not 
provide  all  assignment  data  as  this  involves  AVDO  input. 
REMIS  also  provides  for  direct  input  of  inventory  possession 
and  assignment  data.  REMIS  provides  the  function  of 
"shifting"  data. 

REMIS  has  been  keeping  Inventory  Possession  information  since 
EIMSURS  was  fielded  in  1990.  REMIS  has  euid  will  continue  to 
"keep  up"  with  possession  changes.  Reconciliation  of  CAMS  and 
REMIS  has  been  con^leted  for  RECFU  II. 

PAGE  V-5  PARA  A.3.a 

The  report  specifies  CAMS  and  TICARRS  have  the  caped>ility  for 
direct  entry  of  equipment  identification,  etc.  REMIS  supports 
on-line  entry  of  inventory  information  euid  provides  on-line 
feedback  to  promote  data  accuracy.  REMIS  maintains  both 
current  eind  historical  inventory,  inventory  assigxmient  and 
inventory  possession  information.  Air  Staff  is  also  a  primary 
user  of  REMIS  inventory  data.  TICARRS  does  not  support  Air 
Staff  in  this  manner. 

The  report  states  that  "REMIS  does  not  keep  up  with  possession 
changes".  REMIS  does  not  have  a  throughput  problem  with  CAMS 
EIMSURS  data. 

PAGE  V-5  6  V-6  PARA  b 

Status  for  ATE  has  been  availcdsle  in  CAMS  since  its 
predecessor  (MMICS)  enedsled  the  function  in  1980.  It  has  not 
received  wide  spread  use  because  it  was  intended  for  use  by  F- 
15  and  F-16  back  shops  which  continue  to  use  TICARRS.  NOTE: 
The  CAMS  ATE  functions  are  based  on  the  ATE  functions  in 
TICARRS. 


PAGE  V-6  PARA  A.3.b 

The  report  states  that  edits  pushed  back  to  the  originator  for 
verification  or  correction  are  singly  dropped  after  an 
interval  of  time  whether  or  not  the  verification  or  correction 
has  been  done.  This  is  not  true.  The  original  version  of 
RECFU  II  fielded  in  1992  did  have  an  expiration  period,  for 
returned  transactions;  however,  the  currently  fielded  version 
does  not  delete  esqpired  treuisactions .  All  transactions  remain 
on-line  until  corrected  by  the  originator. 


IV-21 


PAGE  V-6  PARA  A.3.C 


Not  only  are  the  MAJCOMS  primary  users,  but  so  is  the  Air 
Staff.  TICARRS  does  not  provide  utilization  information  to 
the  Air  Staff.  Accurate  utilization  information  is  extremely 
important  since  Major  Command  budgets  are  based  upon  actual 
flight  hours.  Air  Force  personnel  must  determine  if  the 
TICARRS  utilization  function  requires  complete  redesign  or  if 
it  can  be  corrected  to  report  accurate  utilization 
information.  The  cost  for  this  determination  and  any  cost 
associated  with  correcting  deficiencies  is  not  included  in 
Paragraph  VI I. C. 

REMIS  accepts  allocated  flying  hour  data  from  Air  Staff  and 
provides  reports  comparing  allocated  vs.  actual  flying  hour 
data.  This  report  (K002)  is  one  of  the  most  inportant  outputs 
of  CAMS/REMIS.  TICARRS  does  not  support  this  function. 

PAGE  V-6  Sentences  1  and  2 

CAMS  has  a  subsystem  to  track  the  status  of  ATE.  It  is  called 
ATERS.  The  statement  that  "this  CAMS  subfunction  is  not 
availeible  in  CAMS"  is  incorrect. 

PAGE  v*6  PARA  2  Last  sentence 

This  sentence  is  incorrect.  Errors  pushed  back  to  CAMS  from 
REMIS  for  correction  are  not  dropped  after  an  interval  of 
time . 

PAGE  V-6  PARA  C 

Second  Sentence:  "Touch-dovm  lemdings".  All  landings  are 
"touch-down".  What  is  probaUaly  meemt  here  is  "touch  and  go 
landings " . 

The  report  lumps  inventory,  status  auid  utilization  functions 
into  one  pot,  assuming  that  covers  all  equipment.  TICARRS 
only  provides  these  functions  for  aircraft  and  does  not 
provide  inventory  possession  for  ATE,  their  genesis  equipment. 
Inventory  possession,  status  and  utilization  for  trainers 
(flight  simulators,  mission  simulators,  etc.)  are  not  covered 
in  TICARRS  and  this  type  of  reporting  for  C-E  is  so  coirpletely 
different  from  other  equipment  as  to  be  a  major  function  loss 
all  by  itself. 

In  the  paragraph  on  utilization  and  the  comments  about 
accumulated  flight  hour  problems,  it  would  have  been  very  easy 
to  compare  CAMS  &  TICARRS  to  see  where  the  problems  arose, 
since  CAMS  was  maintained  in  its  entirety  during  the  OA.  The 
plans  and  scheduling  sections  used  CAMS  during  the  OA  to 
forecast  time  change  requirements  since  TICARRS  is  unable  to 
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perform  this  function. 

The  last  paragraph  on  this  page  alludes  to  a  problem  ACC 
personnel  have  on  reporting  flying  hours  to  Air  Staff.  This 
is  misleading  since  the  only  source  of  flying  hour  reporting 
to  the  Air  Staff  is  REMIS.  Since  flying  hours  are  used  for 
budgetary  considerations,  ACC  makes  certain  that  data  in  REMIS 
is  accurate  before  the  report  goes  to  the  Air  Staff. 

PAGE  V-7  PARA  A.3.f 

REMIS  does  include  Debriefing  discrepancy  narratives. 

PAGE  V-9  PARA  A.3.g 

REMIS  currently  receives  both  Discrepamcy  and  Corrective 
Action  narratives  from  CAMS.  REMIS  currently  provides  queries 
which  display  these  narratives  to  REMIS  users.  REMIS  users 
may  also  access  these  narratives  through  REMISTALK.  The 
report  discussion  on  narratives  is  incorrect. 

PAGE  V-9  PARA  1 

The  statement  that  CAMS  does  not  automatically  calculate  TCI 
due  dates  and  TICARRS  does,  is  wrong.  TICARRS  does  not  and 
CAMS  does.  CAMS  has  been  automatically  calculating  TCI  due 
dates  since  its  predecessor  (MMICS)  enaibled  this  function  in 
1974.  CAMS  not  only  automatically  calculates  due  dates,  it 
also  forecasts  time  change  item  due  dates  five  years  in 
advance,  by  quarter,  which  is  a  rec[uirement  of  T.O.  00-20-9, 
a  function  TICARRS  does  not  perform. 

PAGE  V-9  PARA  3 

Two- level  maintenance.  The  report  indicates  that  all 
intermediate  work  will  be  performed  at  the  depot.  The  concept 
is  only  envisioned  for  those  LRU' s  that  must  be  tested  on 
Automatic  Test  Equipment  (ATE) ;  for  those  hundreds  of  items 
not  requiring  ATE  support,  the  work  will  continue  at  the 
intermediate  level. 

A-lO's  use  some  of  the  same  avionics  LRU's  as  the  F-16,  but 
TICARRS  cannot  track  them  at  either  base  level  or  depot  level . 

The  statement  edjout  the  B-1  contractor  creating  their  own  data 
base,  rather  than  rely  on  REMIS,  is  inaccurate.  The  B-1  CSAS- 
CDB  was  created  in  1985  (pre-REMIS)  to  track  configuration 
data,  not  maintenance  history  data.  REMIS  GCSAS  absorbs  the 
functions  of  the  B-1  CSAS  and  replaces  it. 

The  statement  that  other  contractors  need  the  data  in  D056, 
because  it  is  not  availadsle  from  REMIS,  is  incorrect.  REMIS 
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has  converted  all  On- Equipment  information  from  D056  into 
REMIS  and  D056  currently  receives  the  same  data  from  CAMS  that 
REMIS  receives. 

PAGE  V-12  PARA  2 

The  report  seems  to  reflect  a  lack  of  familiarity  with 
part /serial  number  tracking.  It  is  possible  to  have  actual 
duplicates  of  part /serial  numbers  in  the  inventory.  This  does 
not  cause  a  CAMS  unit  at  base  level  a  problem  (CAMS  does  not 
allow  duplicates  in  the  same  data  base)  .  This  problem  becomes 
more  serious  when  a  system  deals  with  multiple  types  of 
aircraft  (something  TICARRS  does  not  have  a  problem  with, 
since  they  only  deal  with  the  F-15  and  F-16  aircraft.) 

GCSAS  incorporates  the  CAGE  (Contractor  cuid  Government  Entity) 
in  its  configuration  tracking  order  to  alleviate  the  problem 
of  duplicate  part/serial  numbers. 

PAGE  V-13  PARA  2 

REMIS  is  currently  developing  block  number  tracking.  While 
TCTO's  for  F-16'e  may  be  issued  by  block  number,  they  are 
scheduled,  con^leted  and  reported  by  individual  aircraft 
serial  nu^er. 

No  other  aircraft  fleet  manages  by  block  number  as  P-l6's  do. 
All  aircraft,  including  F-16's,  have  been  managed  in  CAMS  for 
several  years  without  degrading  performance  or  operations. 

PAGE  V-14  PARA  p 

TICARRS  and  SDS  are  not  the  same  system.  SDS  was  based  on 
TICARRS,  but  there  is  currently  no  basis  for  coiiq>arison . 

PAGE  V-16  PARA  B 

The  scope  deficiencies  of  TICARRS  are  only  partially 
addressed.  Scope  concerns  such  as:  interfaces,  master  edit 
and  validation  tables,  communication  methods,  such  as  FTP, 
SMTP,  etc.  ,  transaction  voliunes,  increased  data  relationships, 
increased  user  voliune,  increased  data  size,  increased  data 
con^atibility,  edit  con^lexity,  increased  overall  system 
complexity  and  its  impact  on  user  friendliness,  increased 
training  requirements,  etc.  are  not  addressed.  Also  not 
addressed  is  the  intact  of  the  additional  functions  on  the 
existing  TICARRS  system.  For  exanple,  most  TICARRS  reports 
require  the  user  to  enter  EQUIPMENT  TYPE,  a  two  position  field 
with  the  first  position  representing  Series  (e.g.:  the  E  of 
an  F015E)  and  the  second  position  representing  the  first  non¬ 
zero  digit  of  block  number  (e.g.:  1  would  be  block  010)  .  All 
TICARRS  reports,  utilizing  the  EQUIPMENT  TYPE  data  element. 
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require  modification  if  TICARRS  is  expanded  to  support  other 
aircraft.  This  is  true  because  Series  is  not  unique  across 
multiple  Mission  and  Designs.  Another  exanple  is  TICARR's  use 
of  the  term  ORGID.  Sometimes  it  represents  geographic 
location  indicator  and  other  times  it  represents  the  65-110 
organization  number.  Organization  number  is  not  unique  across 
commands.  All  TICARRS  reports  using  shorthand  data  fields  of 
this  type  require  significant  modification  to  operate  if  the 
scope  is  expanded.  NOTE:  The  TICARRS  data  base  has  the  same 
problem  as  the  reports  and  will  also  need  to  be  redesigned. 
The  cost  associated  with  these  necessary  modifications  is  not 
included  in  PARA  VII.C. 

PAGE  V-17  PARA  B  Last  SUBPARA 

The  report  states,  "REMIS  has  no  inherent  advantage  over 
TICARRS".  It  goes  on  to  state,  "TICARRS  currently  contains 
important  narrative  information  REMIS  does  not".  REMIS 
gathers  narrative  information  under  DDN  processing. 

PAGE  V-21  PARA  1 

The  performance  data  listed  in  Tatble  V-2  for  TICARRS,  Table  V- 
3  for  REMIS  and  Taible  V-4  for  CAMS  does  not  use  comparable 
topics.  On  report  page  V-20,  the  last  sentence  states,  "The 
response  time  data  presented  in  this  report  is  limited  to  the 
central  computer  processing  time  for  each  transaction." 
However,  this  report  penalizes  the  CAMS  response  time  by 
including  the  data  transmission  communications  delays. 

Based  on  the  number  of  transactions  a  TICARRS  user  must  enter 
daily  versus  a  CAMS  user  to  accomplish  one  day's  work,  there 
is  evidence  that  TICARRS  transactions  are  smaller  and  simpler 
to  process.  TICARRS  users  must  enter  four  transactions  for 
every  one  entered  by  a  CAMS  user.  In  other  words,  the  four 
transactions  required  by  TICARRS  at  3  seconds  each  means  the 
TICARRS  user  will  take  12  seconds  to  do  what  a  CAMS  user  can 
do  in  4.56  seconds  (see  first  sentence  on  Page  V-25) . 

For  "...the  second  type  of  transaction  (more  conqplex  and 
lengthy  output)",  the  TICARRS  transactions  were  not  measured. 
Observations  were  accepted  by  the  IDA  study  team.  A  recent 
review  of  standard  TICARRS  reports  shows  that  they  provide 
less  functionality,  scope,  and  complexity  than  similar  REMIS 
report  transactions.  Additionally,  the  relatively  small  size 
of  the  TICARRS  data  base  (22M  records)  vis  a  vis  the  REMIS 
data  base  (over  lOOM  records)  provides  TICARRS  a  report 
processing  response  time  advantage  which  would  vanish  if 
TICARRS  atten^ted  full  CAMS/REMIS  functionality  and  scope. 

Furthermore,  not  only  were  no  response  time  measurements  taken 
for  TICARRS  standard  reports,  "...there  were  no  measurements 
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of  the  number  and/or  throughput  of  standard  reports"  taken. 


PAGE  V-22  PASA  1 

This  section  notes  that:  "TICARRS  would  need  to  be 
substantially  increased  (about  3  times  more  power)  to  maintain 
satisfactory  response."  However,  Page  VII- 18  states  that: 
"Con5)uter  power  must  be  4.4  times  the  existing  level. "  A  more 
accurate  sizing  of  the  CAMS/RBMIS  equivalent  TIOJIRS  con^uter 
is  found  under  the  comments  for  Page  VII -18,  Mainfrcune  Sizing 
for  TICARRS.  That  sizing  shows  that  the  TICARRS  con?)uter 
would  need  to  be  18.66  times  its  existing  size,  not  4.4. 


PAGE  V-22  and  V-23  PASA  2 

While  current  measurements  show  that  the  \HQ1  CPUs  are 
operating  at  60%  capacity,  other  resources  including  disk,  I/O 
channel,  interprocessor  bus,  amd  memory  consumption  are  well 
below  the  35%  utilization  ramge.  Implementation  of  the  system 
performance  team's  pleuis  and  recommendations  will 
significamtly  improve  system  balamcing .  These  activities  will 
result  in  improvements  in  system  performauice  auid  response 
time.  In  addition,  80%  of  operations  functions  amd  all  of 
system  test  fimctions  have  been  moved  from  \HQ1  to  \HQ2. 

System  timing  continues  to  receive  high  priority  and  is 
accomplished  daily.  The  NCC  closely  monitors  CPU  utilization 
and  moves  processes  as  required  to  achieve  CPU  balamce.  The 
system  administrator  analyzes  system  performance  daily  and 
applies  tuning  techniques  when  amd  wherever  possible  to 
improve  performance. 

PAGE  V-23  TABLE  V-3 

The  TICARRS  performamce  data  did  not  provide  similar 
information  to  the  REMIS  performamce  data.  A  comparison  based 
on  dissimilar  data  is  not  valid.  REMIS  total  transactions  per 
day  are  not  provided. 


PAGE  V-23  PARA  1 

Given  the  historical  number  of  standard  amd  REMISTALK  reports 
requested  daily  by  REMIS  users,  the  Septemiber  1992  projections 
of  the  number  of  reports  anticipated  is  much  too  high.  No 
noticeadile  upward  trend  is  evident  in  the  number  of  either 
type  of  report  during  the  first  six  months  of  1993.  The  only 
significant  increase  in  users  will  occur  when  6CSAS  becomes 


fully  operational,  but  those  users  alone  are  not  expected  to 
account  for  anywhere  near  a  ten- fold  increase  in  reports 
generated  daily. 

PAGE  V-25  PARA  2  AMD  FIGURE  V-3 

The  second  paragraph  on  this  page  states,  "While  edsout  33%  of 
the  CAMS  bases  have  average  response  times  of  3  seconds  or 
less,  67%  have  average  response  times  of  greater  thaui  3 
seconds."  To  state  this  same  information  cmother  way,  it  is 
true  that  about  33%  of  the  CAMS  bases  have  average  response 
times  of  3  seconds  or  less,  62%  of  the  bases  have  response 
times  of  4  seconds  or  less,  and  77%  of  the  bases  have  5  second 
or  less  response  times.  Since  the  software  is  the  saime  for 
all  bases,  the  difference  is  in  the  SBLC  operation  at  some 
bases.  As  the  report  states  on  page  V-37,  "The  move  to 
Regional  Processing  Centers  (RPC)  will  tend  to  iit5)rove 
operational  control  and  discipline. . . " . 

PAGE  V-29  PARA  C 

EIMSURS  functions  include  utilization. 

GCSAS  functions  include  TCTO  management. 

PAGE  V-32  PARA  C.l.c(3)  2nd  SUBPARA 

The  statement  that  the  TICARRS  data  base  is  structured  to 
cover  the  entire  (world-wide)  fleet  of  weapons  systems  similar 
to  REMIC  is  incorrect.  The  inplication  that  TICARRS  current 
data  base  structure  can  support  multiple  Mission  Design  Series 
(MDS)  for  equipment  other  thaui  F-16  euid  F-15E  is  misleading. 
TICARRS  data  base  structure  relies  heavily  on  a  field  called 
EQUIPMENT  TYPE  which  cannot  be  used  across  MDS  where  Series  is 
not  unique. 

PAGE  V-32  PARA  4 

This  paragraph  states,  "One  of  the  most  frequent  conplaints  by 
CAMS  users  is  the  requirement  to  re-enter  data  when  moving  to 
a  new  screen  in  the  same  process.  This  occurs  in  TICARRS  as 
well ,  but  it  does  not  seem  to  be  as  great  a  concern  to  users . " 
If  CAMS  euid  TICARRS  both  have  the  same  problem,  why  is  it  a 
problem  for  CAMS  and  not  "as  great  a  concern"  for  TICARRS? 

PAGE  V-33  PARA  C.l.d 

As  the  report  points  out,  REMIS  utilizes  a  prototyping 
methodology  for  requirement  definition  when  designing  large 
program  components.  Prototyping  was  utilized  on  GCSAS  with  a 
high  degree  of  success.  The  report  failed  to  address  the 
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other  requirement  definition  methods  eir5)loyed  by  REMIS.  In 
determining  requirements  for  the  Communication- Electronics 
subfunction  of  EIMSURS,  REMIS  designers  interacted  directly 
with  representatives  of  the  C-E  community  from  each  of  the 
Major  Commands.  This  direct  user  involvement  enabled  REMIS  to 
develop  a  design  that  not  only  met  the  user  needs  but  also 
standardized  C-E  processing  across  the  Air  Force.  REMIS 
selects  the  particular  requirement  definition  approach  to 
utilize  depending  upon  the  scope,  con^lexity  and  nature  of  the 
project.  The  major  factor  included  in  all  REMIS  requirement 
definition  approaches  is  direct  user  involvement. 

PAGE  V-36  PARA  C.l.f 

There  is  a  wrong  way  to  manage  the  activity,  and  that  is  the 
attempt  to  agree  on  requirements,  which  will  become 
contractually  binding  by  having  tens  of  Contractor  Field 
Representatives  interface  with  perhaps  hundreds  of  functional 
users.  This  has  never  been  worked,  never  will,  cmd  always 
results  in  enormous  contractual  amd  financial  problems. 

The  report  estimates  that  the  TICARRS  con5)uter  system  capacity 
will  need  to  be  4  to  4.5  times  more  powerful.  This  estimate 
is  severely  understated.  The  report  specifies  the  estimate  is 
based  on  CAMS  transaction  volxime.  REMIS  volume  should  also 
have  been  considered,  especially  in  the  area  of  interfaces  and 
user  reports.  The  additional  functions  TICARRS  would  be 
required  to  support  increases  the  number  of  linking  or 
relating  data  base  records  needed.  With  the  increase  in  data 
maintained  on-line,  each  report  will  be  recnaired  to  read 
through  more  information  to  develop  the  output.  Therefore, 
each  report  will  consume  more  system  resources  than  it  does 
currently.  Increased  trajisaction  volvune  requires  increased 
system  resources  dedicated  to  functions  such  as  roll  forward 
and  roll  back.  When  factors  such  as  these,  emd  those 
specified  in  comments  addressing  Page  V-16,  Paragraph  B  are 
considered,  it  becomes  obvious  that  system  size  does  not  have 
a  linear  relationship  with  transaction  volume  and  functions, 
but  is  exponential. 

PAGE  V-36  PARA  F  2nd  SUBPARA 

In  the  second  paragraph,  the  author  makes  the  statement  that, 
based  on  treuisaction  volxime,  the  TICARRS  computer  system  would 
have  to  be  4  to  4.5  times  more  powerful.  This  is  understated 
by  a  factor  of  5.  Transaction  volume  alone  does  not  determine 
the  size  of  a  computer  system,  and  the  fact  that  CAMS  supports 
109  locations  while  TICARRS  only  supports  35,  alone  makes  size 
a  factor  of  3 .  To  suppose  that  adding  in  the  additional 
functions  of  CAMS  and  the  ability  to  handle  all  the  thousands 
of  additional  equipment  types,  only  increases  workload  by  l  to 
1.5  is  unrealistic. 
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PAGE  V-36  PASA  C.l.f  3rd  SHBPARA 

The  topic  is  system  technology  and  performance  summary.  The 
report  addresses  extending  the  REMIS  standard  and  REMISTALK 
report  capabilities  to  allow  a  broad  scope  of  data.  This 
recommendation  is  misplaced  as  it  does  not  relate  to  system 
technology  or  performance.  No  mention  of  the  limited  TICARRS 
report  scope  is  included  and  is  as  relevant  as  the  REMIS 
report  scope. 

PAGE  V-37  PAHA  1 

This  paragraph  contradicts  itself.  First,  it  says,  "Where 
SBLC  support  is  good,  CAMS  operation  and  availability  is 
good."  Then  it  states,  when  the  move  to  the  Regional 
Processing  Centers  is  con?)leted,  "CAMS  software  reliability 
needs  to  be  excunined  to  ensure  it  is  not  the  basis  of 
availadDility  problems." 

PAGE  V’37  PARA  2 

CAMS  receives  all  the  S2une  edit/validation  tables  that  REMIS 
uses.  CAMS  does  reject  99%  of  incorrect /invalid  data  when  it 
is  entered.  CAMS  must  rely  on  REMIS  to  validate  those 
conparative  few  transactions  that  require  views  of  data  for  cui 
entire  MDS,  that  is,  configuration  data. 

PAGE  V-38  PARA  A 

See  Comments  for  Pages  II -1  and  II -9  (Above) . 


PAGE  v-38  PARA  a  "REMISTALK  has  poor  training": 

To  date,  REMISTALK  training  has  trained  151  students.  The 
students  have  consistently  rated  the  instructor,  course 
materials  and  overall  course  content  cdjove  average  to 
excellent  on  the  REMISTALK  critiques.  Numerous  comments  have 
been  provided  by  students  citing  the  outstemding  efforts  of 
the  instructor,  the  educational  value  of  the  course  materials 
and  the  excellent  course  content  and  information  flow. 
Students  with  NO  prior  experience  have  stated  that  they  felt 
confident  they  could  now  easily  layout  and  build  REMISTALK 
reports.  Extensive  one-on-one  training  with  each  student  is 
provided  during  these  training  sessions  euid  students  are 
provided  with  ample  time  and  opportxinity  to  layout/build 
REMISTALK  reports  tailored  for  their  individual  needs. 

Attachment  5  is  a  copy  of  the  critique  form  each  student  must 
complete  upon  graduation.  If  the  students  rated  an  area  below 
4  (3.5  is  average),  it  is  included  in  the  category  "other". 
You  will  find  that  95%  of  the  students  consistently  rated  all 
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areas  at  a  4  or  ahave. 


PAGE  V-38  PAEA  C.2.a 

The  report  lists  user  criticisms  of  REMIS  under  the  heading  of 
I4AJC0MS  (ACC)  .  ACC  is  only  one  MAJCC»1.  Were  other  MAJCC»4s 
polled?  Questionnaires  should  address  specific  system 
functions  and  not  allow  for  ambiguous  statements,  such  as: 
"REMIS  is  alow" . 

PAGE  V-39  PASA  A 

The  report  states  that, "REMIS  has  poor  training". 

Litton  Computer  Services  (LCS)  has  developed  and  conducted 
three  REMIS  courses  for  the  Air  Force  between  1987  and  1993: 

REMIS  EIMSURS  End  Users  Course  -  E10SC4925-314 
REMIS  PPS  End  Users  Course  -  E10SC4925-000 
REMIS  GCSAS  End  Users  Co  rse  -  E10SC4925-009 

Each  course  was  developed  and  approved  per  contractual 
requirements  and  sovind  training  practices  for  adult  learning. 
Air  Training  Command  (ATC)  approved  each  course  prior  to  the 
course  being  presented  by  LCS  to  the  REMIS  end-users. 

To  date,  REMIS  End-User  training  has  trained  over  1,666 
students.  Ntimerous  comments  have  been  provided  by  students 
citing  the  outstanding  efforts  of  the  instructor,  the 
educational  value  of  the  course  materials  and  the  excellent 
course  content  and  information  flow. 

The  report  states  that  Contractors  cited  poor  training  in 
REMIS.  Out  of  the  1,600  plus  students  who  have  attended  REMIS 
courses  (EIMSURS,  PPS,  or  GCSAS)  only  6  have  been  Contractors. 
These  six  have  rated  all  areas  on  the  student  critiques  with 
5  or  6  ratings  (see  Attachment  6  for  a  san^le  of  course 
critique) . 

Again,  Attachment  6  is  a  copy  of  the  critique  form  each 
student  must  complete  upon  graduation.  If  the  students  rated 
an  area  below  4  (3.5  is  average),  it  is  included  in  the 

category  "other".  Again,  you  will  find  that  95%  of  the 
students  consistently  rated  all  areas  at  a  4  or  above. 


PAGE  V-39 

The  report  states  that,  "Contractors  have  the  following 


IV-30 


opinions:  They  prefer  working  with  raw  tapes  of  Product 
Performance  Data." 

During  the  time  frame  of  October  1992  through  April  1993,  more 
than  a  dozen  Contractors  who  currently  receive  D056  data  on 
tape  were  contacted.  Each  was  offered  the  choice  of 
continuing  to  receive  tapes  of  Product  Performance  Data  or 
log-on  to  REMIS  to  view  their  data  or  have  the  data  sent 
across  DDN  to  their  computers. 

None  of  the  Contractors  contacted  elected  to  continue 
receiving  data  on  tapes. 

PAGE  V-45  PAKA  D 

The  report  states  that,  "Discussions  with  MAJCOM  personnel. 
Weapon  System  SPO's  and  contractor  personnel  all  favored 
TICARRS " . 

This  statement  makes  it  seem  it  was  all  MAJCOM,  all  SPO's,  and 
all  Contractors,  while  in  fact,  it  was  1  MAJCOM  (ACC) ,  1  or  2 
SPO's  (F-15  and  F-16),  and  a  handful  of  contractors.  This 
group  has  been  using  TICARRS  for  over  10  years  and  are  very 
familiar  with  it.  Since  REMIS  is  new,  resistance  should  be 
expected. 


PAGE  V-46,  PARA  D.l 

The  report  contends  that  data  accuracy  is  am  intrinsic  problem 
with  CAMS/REMIS  due  to  the  system  architecture.  The  reader  is 
lead  to  believe  that  fleet-wide  edit  and  validations  cannot 
occur  in  real-time  mode.  Currently,  CAMS/REMIS  has  software 
in  testing  that  enaibles  a  CAMS  user  to  query  the  REMIS  data 
base  in  on-line  real  time  mode.  This  is  the  first  step  in 
providing  CAMS  base  level  users  with  fleet-wide  data 
validation  at  the  time  the  data  is  initially  entered  into 
CAMS.  This  capability  is  not  addressed  in  Paragraph  VII  but 
should  be.  It  is  a  vicd:>le  alternative  to  a  single  centralized 
system  architecture  and  represents  a  much  smaller  investment 
to  the  Air  Force  then  does  the  expansion  of  TICARRS. 

PAGE  V-47  PARA  2 

The  difference  in  rounding  flying  hours  has  been  corrected  in 
EIMSURS.  REMIS  and  CAMS  round  flying  hours  the  sarnie  way. 


PAGE  V-52 

It  becomes  clear  when  it  is  recognized  that  the  same 
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maintainers  who  entered  data  directly  into  TICARRS  entered 
data  directly  into  CAMS.  The  difference  is  the  Contractor 
Representatives  for  TICARRS  doing  the  maintainer's  job  for 
them. 

PAGE  V-52  2IID  SUBPARA  AMD  FIGURE  V-8 

The  last  sentence  states,  "It  is  not  clear  from  existing  data 
whether  the  trend  is  showing  a  decline  or  remaining  stable 
within  the  20  to  40%  range."  A  look  at  Figure  V-8  clearly 
shows  a  steady  decrease  in  errors  from  the  PPS  IOC  through 
April  1993 . 

PAGE  V-54,  PARA  D.3 

If  there  are  no  problems  with  TICARRS  security  procedures  as 
stated  in  this  paragraph,  why  would  "appropriate  security 
procedures  have  to  be  developed"? 

The  report  states  that  REMIS  has  a  data  security  problem  in 
that  users  are  able  to  update  configuration  information  on 
other  than  the  weapon  system  the  user  is  responsible  for.  The 
REMIS  GCSAS  software  was  modified  in  Jainuary  1993  to  provide 
user  access  restriction  by  Equipment  Designator  (KC135  vs 
C141)  for  all  approved  configuration  data.  This  modification 
is  scheduled  to  be  fielded  prior  to  the  first  weapon  system 
being  in^lemented  within  GCSAS. 

PAGE  V-64,  PARA  E.2 

Both  CAMS  and  TICARRS  utilize  a  network  data  base.  The  CAMS 
discussion  describes  a  network  data  base  as,  "...very 
difficult  to  manage  and  control  at  the  CSDB  level".  The 
TICARRS  discussion  describes  a  network  data  base  as, 
"...suited  to  handling  many  data  base  activities  rapidly". 
Does  the  report  conclude  that  data  bases  are  an  asset  or  a 
handicap? 

PAGE  VII -3  First  Paragraph 

This  paragraph  iirplies  that  the  CAMS  portion  of  the  SBLC  is 
unknown.  The  IDA  study  team  determined  to  use  50%  "...as  the 
factor  to  apportion  SBLC  and  RPC  variable  costs  to  CAMS." 
Interestingly,  the  IDA  study  team  had  previously  published  in 
Table  V-4,  Page  V-25,  that  the:  "Percent  of  SBLC  use  due  to 
CAMS"  was  specifically  43%.  Mhy  was  the  percentage  raised  for 
projecting  future  costs? 

This  same  paragraph  estimates  that  the  fixed  SBLC  and  RPC 
costs  are  20%  and  the  varicdsle  costs  are  80%.  Mo  basis  of 
estimate  is  provided  to  justify  this  estimate.  It  appears  to 
be  based  on  the  convenience  of  the  "80/20  rule"  instead  of 
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being  based  on  factual  data. 

PAGE  VZI-4  TABLE  VII -1 

Justification  for  2.5.1  Adjustment  to  Software  Maintenance 
Costs 

As  described  on  page  Vli-7  and  VII- 8,  the  software 
applications  maintenance  is  computed  as  45  people  per  year  at 
$43,099  per  person.  This  equals  $1.94M  per  year  or  $19.39M 
total  for  the  ten -year  period.  The  last  sentence  in  the  first 
paragraph  on  Page  VII -8  should  be  deleted,  since  the  conputer 
support  and  system  integration  costs  are  listed  separately  as 
Line  Items  2.5.4  and  2.5.5  respectively. 


PAGE  VII -4  AMD  VII -32 

Comparison  of  Table  VII- I  and  Table  VII- lO  shows  some 
disparities  in  the  costs  for  ALT  2. 

1.1.1  -  Given  this  is  mid- July,  will  all  RPC  H/W  costs  be 
saved? 

1.2.1  -  It  would  be  in^ossible  to  stop  modifications  to 
CAMS  for  three  years  even  if  it  were  phased  out. 
Statutory  and  mission  changes,  new  weapon 
systems/equipment  and  user  pressure  will  force  continued 
modifications. 

2.3.2  -  Cost  for  Base  Representatives  of  $5.31M/yr  for 
CAMS  (TcUole  VII -1)  euid  "0"  in  ALT  2  doesn't  pass  logic 
test.  If  these  people  are  not  on-board  now,  then  $5.31M 
worth  won't  be  in  place  on  1  October  1993.  Similarly,  if 
they  are  already  in  place  (i.e.  maybe  this  refers  to  CAMS 
data  base  meuiagers)  then  they  will  not  all  be  gone  on  l 
October  1993,  and  there  should  be  a  cost  to  ALT  2. 

PAGE  VII -4  and  VII-6,  System  Integration  Test 

Costs  are  included  for  CAMS  independent  system  integration 
test  but  are  not  included  for  TICARRS  (see  Page  VII -17) . 

PAGE  VII- 5,  Approved  Functional  Modifications 

The  cost  to  provide  additional  functional  modifications 
(CSRDs)  to  CAMS  was  included  in  the  CAMS  cost  estimate.  These 
new  capabilities  were  not  evaluated  to  determine  if  TICARRS 
required  similar  enhancements  and  the  costs  of  these  are  not 
included  in  the  TICARRS  cost  estimates. 
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PAGE  VII -6 


Cost  are  placed  against  CAMS  for  new  weapon  system  support  for 
the  F-22,  IMIS,  etc.  However,  similar  costs  are  not  placed  on 
TICARRS  for  the  F-22  (see  Ted>le  VII-6,  Page  VII -24) . 

PAGE  VII-6  (CAMS  Extensions  and  New  Weapon  Support) 

There  is  no  corresponding  discussion  for  TICARRS,  and  new 
weapon  systems  (i.e.,  F-22)  are  not  included  in  the  data  base 
initialization  costs  (Tsdile  VII-6)  . 


PAGE  VII -7  PARA  2E 

Included  in  the  Application  Maintenance  Costs  for  CAMS  were 
the  following:  hardware  maintenance,  conputer  operations,  and 
supplies  in  addition  to  "software  maintenance"  costs.  These 
were  not  included  in  the  TICARRS  software  maintencuice  cost 
estimate.  (see  Page  VII-30,  Para  2e) . 

PAGE  VII-10  TABLE  VII-2  1.2  AND  VII-11  PARA  4.b 

Improving  Performance;  The  report  estimate  that  15  staff 
years  for  three  years  will  be  required.  LCS  estimates  that  no 
more  than  10  staff  years  for  three  years  would  be  considered 
for  performance  improvement  as  a  non-recurring  cost.  These 
would  consist  of  application  emalysts,  system  euialysts,  and 
programmers.  Costs  would  be  10  people  times  $115,200  per 
staff  year  which  equals  $1.15M  per  year,  or  $3.46M  for  three 
years . 

Improving  REMISTALK;  The  report  estimates  10  people  for  three 
years  to  expand  cuid  in^rove  REMISTALK.  LCS  plans  to  use  one 
person  over  the  ten-year  period  to  perform  all  REMISTALK 
related  maintenance  eind  enhancements.  This  will  cost  $115,200 
per  year,  or  $1.15M  for  the  ten-year  period.  It  may  be  more 
appropriate  to  consider  this  a  recurring  cost. 

Initializing  Data  Base:  With  each  REMIS  subsystem  placed  in 
production,  the  data  loaded  into  that  svibsysten/ s  respective 
data  base  tables  has  been  checked  by  existing  staff.  This 
tas)c  for  GCSAS  will  be  handled  as  part  of  normal  operations  by 
staff  already  accounted  for  under  recurring  costs.  This  task 
is  much  smaller  than  the  report  assumes  and  will  be  handled  by 
existing  staff  during  pre  and  post  in^lementation  of  GCSAS  as 
part  of  the  recurring  operations  for  REMIS.  No  additional 
non-recurring  expense  is  required. 
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PAGE  VII -13  PASA  B  AND  C 

LCS  is  projecting  21  personnel  to  operate  the  Network  Control 
Center  in  FY94. 

The  ALCs  require  no  manning.  These  sites  are  operated  from 
the  NCC  in  a  "lights  out"  mode.  No  manning  is  projected  for 
the  ALCs.  User  support  and  operation  of  the  computer  systems 
are  co- functions  of  the  Network  Control  Center  and  are  part  of 
the  21  personnel  projected  above  for  FY94.  Although  LCS 
manpower  projections  might  have  to  be  increased  with  a  large 
hardware  upgrade,  in  no  way  do  we  see  requirements  for  the  47 
people  as  projected  by  IDA. 

PAGE  VII -16 

The  assunptions  presented  are  unreasonable.  If  it  is  the 
government's  intent  to  compare  information  systems  which  meet 
all  of  the  users'  requirements,  no  favoritism  should  be  shown 
to  either  system.  Since  CAMS/REMIS  must  go  through  a  MAISRC 
review,  TICARRS  should  also.  Since  OT&E  for  CAMS/REMIS  tests 
functionality  and  scope  required  for  all  weapons  systems,  OT&E 
for  TICARRS  should  test  the  same  functionality  and  scope 
requirements  specified  for  the  Air  Force  standard  system  they 
wish  to  replace.  Preferential  treatment  should  not  be  granted 
to  either  system. 

The  implication  that  TICARRS  can  be  enhanced  in  six  months  to 
provide  all  of  the  CAMS/REMIS  functionality  is  unrealistic. 
Omitted,  and  therefore  assumed  to  be  provided,  is  the 
requirement  to  also  provide  all  of  the  scope  of  the  CAMS/REMIS 
in  TICARRS  during  the  six-month  enhancement.  Without  the 
scope  enhancement,  any  con^^arison  is  an  "apples  to  oranges" 
comparison. 

PAGE  VII -17  TABLE  VII -4 

Justification  for  VII-17  1.1.1  Recommended  Adjustment  to 
TICARRS  Non-recurring  H/W  Costs 

Page  VII -5  references  "additional  funding  in  the  out  years  to 
provide  for  hardware  replacement  and  incidental  hardware 
purchases".  Table  VII -l  estimates  this  additional  funding  for 
CAMS  at  $2M.  This  is  approximately  7.2%  of  CAMS  share  of  the 
RPC  replacement  costs  for  each  of  six  years  (FY1995-2000)  and 
5.3%  for  an  additional  two  years  (FY2002-2003) .  The 
percentages  were  determined  as  follows: 

$2M  /  $27.90  X  100  -  7.2% 

$1M  /  $10.75  X  100  -  5.3% 

Similar  "incidental  hardware  purchases"  should  be  applied  to 
TICARRS  costs  in  Table  VII-4. 
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The  adjustment  to  VII -17  1.1.1  follows: 

Sum  the  hardware  purchases  for  1994  and  1995.  Calculating  the 
"additional  funding",  we  determine  that  $4.03M  ($56M  x  7.2% 
*  $4.03M)  is  needed  in  years  1996  through  1999.  Using  the 

IDA  team's  assuitption  (see  page  VII- 5,  second  paragraph  under 
l.a  Hardware)  that  mainfraune  conputer  equipment  will  require 
replacement  after  seven  years  and  that  a  20%  compound  annual 
cost  reduction  will  continue,  we  determine  that  the  current 
TICARRS  computer,  which  was  purchased  in  1990  for  $3M  will  be 
replaced  in  1996  for  $0.8M.  Likewise,  the  equipment  purchased 
in  1994  and  1995  will  be  replaced  in  years  2000  and  2001.  The 
"additional  funding"  for  years  2002  and  2003  are  calculated  at 
5.3%  times  the  stun  of  the  equipment  purchased  in  the  years 
2000  and  2001  {$14.68M  x  5.3%  -  $0.78M). 

Justification  for  page  VII-17  1.4.3  Training. 

Page  VII -27  specifies  that  each  tinit  will  require  three  weeks 
of  training  as  part  of  the  unit  activation.  Costs  associated 
with  student  ladaor  hours  should  be  included  and  are  estimated 
based  upon  the  following: 

Page  v-25  states  that  average  daily  log-on  for  an  average  base 
is  196.  The  report  specifies  107  bases.  Thus: 

107  locations  x  196  log-ons  ■  20,972  users 

20,972  users  9  3  weeks  ea  (120  hrs)  «  2,516,640  hours 

$43,099  salary  ($43,099/1920)  *  $22.45  per  hour 

2,516,640  hours  x  $22.45  per  hour  =  $56,498,568 

This  $56. 5M  in  trainee  costs  will  be  allocated  to  FY1995  and 
FY1996  at  $28.25M  per  year. 

This  training  represents  the  costs  associated  with  the 
transition  of  CAMS -trained  users  to  the  TIC^ARRS  system.  NOTE: 
Training  costs  associated  with  personnel  turnovers  are 
considered  to  be  substantially  the  same  regardless  of  the 
system. 


Justification  for  VII-17  2.2.1  Adjustment  to  Hardware  amd 
Software  Maintenance  Costs 

On  page  VII -28  Paragraph  b  the  report  states  that  the 
".  .  .cost  for  hardware  maintenance  is  estimated  to  be  8%  of  the 
cost  of  the  system  hardware."  "The  cost  of  software  license 
is  increased  to  account  for  the  additional  number  of 
processors  in  the  configuration." 
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Table  VII-4  identifies  the  FY1994  and  1995  mainframe  costs  to 
be  $13. 2M  and  Line  Item  2.2.1  H/W,  S/W,  Utilities  costs  to  be 
$1.89M.  Calculating  the  hardware  portion,  we  get  $13. 2M  x  8% 
=  $1.06M.  To  calculate  the  software  costs  included  in  Line 
Item  2.2.1,  we  subtract  the  hardware  costs  from  the  total 
costs  ($1.89M  -  $1.06M  «  $.83M).  These  costs  are  based  on 
supporting  the  4.4  additional  TICARRS  con^juters  estimated  by 
IDA  to  support  CAMS/REMIS  users. 

To  determine  the  software  license  costs  for  each  computer,  we 
divide  the  software  costs  by  the  number  of  con^juters 
($.83M/4.4  =  $.19M)  . 

Adjusting  the  software  cost  for  the  corrected  number  of 
computers,  we  multiply  the  software  cost  per  corr^uter  ($.19M) 
by  the  corrected  number  of  computers  (18.66)  to  obtain  the 
software  cost  of  $3.5M  per  year. 

Adjusting  the  hardware  cost  for  the  corrected  number  of 
computers,  we  multiply  the  hardware  cost  of  $56M  (18.66  x  $3M) 
by  the  annual  hardware  cost  factor  (8%)  to  obtain  the  hardware 
cost  of  $4.48M  per  year. 

We  assume  that  the  H/W,  S/W,  Utilities  Line  Item  will  remain 
static  throughout  the  ten-year  period  as  was  assumed  in 
costing  the  CAMS/REMIS  estimates. 

Adding  the  hardware  and  software  costs  together,  we  obtain  an 
annual  cost  of  $7.98M  ($3.5M  +  $4.48M  -  $7.98M)  .  This  yields 
a  ten-year  cost  of  $79. 8M. 


Justification  for  VII -17  2.2.2  Civilian  and  Military 
Personnel 

Page  VII- 15  states  that  TICARRS  will  utilize  DPCs,  terminals, 
and  printers  currently  at  various  base  locations,  including 
the  staffing  to  support  equipment  as  outlined  in  the  RPC  plan. 

Page  VII-4  Tcible  VII -l  2.2.3  states  this  cost  is  $11.38M  per 
year.  Page  VII -17  Table  VII-4  1.4  shows  that  unit 
activation  costs  will  be  distributed  equally  across  FY95  and 
FY96.  Page  VII-26  Paragraph  f  states  that  unit  (site), 
"activation  will  be  carried  out  over  a  two-year  period, 
beginning  in  FY1994". 

Based  on  the  specified  unit  activation  schedule,  the  last  unit 
will  be  implemented  during  FY1996;  therefore,  FY1996  Base  Comm 
Operations  should  be  estimated  at  the  full  rate  of  $11.38M. 
The  30%  allocation  of  the  $11.38M  (3.83)  should  begin  in 
FY1994  to  support  the  aforementioned  site  activation  schedule 
and  the  67%  (7.65)  should  be  allocated  to  FY1995. 


IV-37 


PAGE  VII -18  AMD  VII -19 

Mainframe  sizing  for  TICARRS. 

During  the  assessment  period,  an  average  of  44,018  daily 
TICARRS  transactions  were  entered  for  one  base,  Seymour 
Johnson  AFB.  This  number  was  derived  from  averaging  the 
average  transactions  per  day  from  weeks  two  through  six  of  the 
assessment  period  (see  page  V-21,  Tadjle  V-2)  .  Week  one  was 
excluded  due  to  abnormally  low  transaction  rates  resulting 
from  users  gaining  experience  with  TICARRS,  from  an  overloaded 
Data  Communication  Processor  (DPC)  and  from  an  imbalanced 
central  con5)uter  (see  page  V-21,  second  paragraph)  . 

The  daily  trainsactions  from  all  CAMS  bases  averaged  12,166 
transactions  per  day  per  base  (see  page  V-25,  Table  V-4  and 
page  V-26,  Figure  V-4)  .  As  stated  in  the  IDA  report,  some 
data  collected  during  the  TICARRS  Operational  Assessment  had 
to  be  directly  entered  into  CAMS,  since  TICARRS  did  not 
include  all  required  functionality,  such  as  engine  end  item 
data  required  for  CEMS  (see  page  V-11,  paragraph  i) .  If  we 
assume  that  no  more  them  an  additional  ten  percent  of  the 
TICARRS'  44,018  daily  transactions  had  to  be  entered  into 
CAMS,  then  the  TICARRS  assessment  personnel  entered  about 
48,420  tremsactions  per  day.  It  is  interesting  to  note  that 
the  average  CAMS  base  cam  perform  more  functionality  with  a 
broader  scope  than  TICARRS  while  at  the  same  time  entering 
only  one- fourth  the  number  of  tramsactions . 

Based  upon  the  transaction  rates,  it  is  apparent  that  four 
TICARRS  transactions  must  be  entered  for  each  equivalent  CAMS 
transaction.  Therefore,  the  equivalent  ntunber  of  daily  first 
shift  TICARRS  tramsactions  will  be  1.2M  CAMS  tramsactions 
times  4  TICARRS  transactions  per  CAMS  tramsaction  which  equals 
4.8M  TICJ\RRS  transactions  per  day  during  the  first  shift. 

Following  the  calculation  methodology  describe  on  page  VII -19: 

4.0M  CAMS  equivalent  transactions  divided  by  20,000  average 
peak  TICiARRS  transactions  equals  240,  the  transaction 
processing  multiplier. 

240  X  4.6%  -  1104%  of  current  TICARRS  power. 

CAMS  support  requires  1104%  of  power  used  to  support  20,000 
TICARRS  tramsactions  during  a  peak  8 -hour  shift. 

Current  REMIS  processing  uses  the  following  resources: 

\HQ1,  12  VLX  processors  9  57%  ■  6.84  VLX  processors 

\HQ2,  8  VLX  processors  9  15%  =  1.2  VLX  processors 

5  ALCs,  10  VLX  processors  9  15%  «  l , 2  VLX  processors 

Total  9.54  VLX  processors 
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Each  VLX  processor  is  rated  at  3.3  MIPS.  Therefore,  9.54 
processors  times  3.3  MIPS  per  processor  equals  31.5  MIPS  at 
100%  utilization  consumed  for  REMIS  without  the  GCSAS 
subsystem  being  fully  operational.  Adding  25%  to  the  current 
lii.4IS  processor  utilization  for  GCSAS  yields  39.4  MIPS 
rtq^iired  to  operate  the  FOC  REMIS. 

Since  TICARRS  central  computer  is  rated  as  a  36  MIPS  machine, 
109%  of  a  TICARRS  computer  is  required  to  support  REMIS  users 
(39.4  MIPS  /  36  MIPS  -  109%). 

The  TICARRS  power  needed  to  support  the  CAMS/REMIS  system  is 
1213%  (1104%  for  CAMS  plus  109%  for  REMIS) . 

TICARRS  proposes  to  operate  its  computer  at  an  average 
utilization  of  65%  during  pea]c  hours.  Therefore,  the 
increased  size  of  the  TICARRS  conputer  will  be  1213%  /  65%  or 
18.66  times  the  current  TICARRS  computer  resources. 

Cost  of  the  current  TICARRS  computer  is  $3M.  The  cost  of  the 
new  central  conputer  is  $3M  times  18.66  which  equals  $56M. 
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CAMS  also  has  interfaces  with  numerous  systems.  For  example, 
INFOCEN  for  PQDR  data.  The  CAMS  SSO  should  be  identified  and 
the  costs  included  in  the  cost  of  the  TICARRS  solution. 

PAGE  VII-21  TABLE  VII-5 

Some  significant  cost  factors  appear  to  have  been  overloo)ced 
in  the  TICARRS  costing  data; 

Enhancements.  The  analysis  did  not  include  C-E;  this  is 
a  major  enhancement.  If  the  author  feels  it  will  talce 
1.3  staff  years  to  include  AGE,  it  will  ta)ce  13  staff 
years  to  include  C-E,  since  C-E  requirements  are  at  least 
10  times  the  requirements  of  AGE. 

The  analysis  did  not  talce  into  account  how  a  central  data 
system  must  be  modified  to  interface  with  over  109 
separate  supply  systems. 

No  cost  has  been  projected  for  FY94  for  the  correction  of 
the  over  400  Functional  Disconnect  Worlc-Around  Write-Ups 
(FDWWS)  nor  for  Difficulty  Reports  (DIREPS)  discovered 
during  the  SJ  Operational  Assessment  described  on  Page 
IV- 9. 

No  cost  has  been  projected  for  loading  REMIS  data  into 
TICARRS. 
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The  mainfrcune  cost  of  two  expenditures  of  $6.6  million 
each  is  based  on  a  projected  capacity  of  4.4  times  the 
current  level.  The  4.4  times  estimate  is  believed  to  be 
severely  understated-  Please  reference  comments 
addressing  Page  V-36,  Paragraph  C.l.f.  Costs  associated 
with  additional  hardware  above  the  4.4  times  should  be 
reflected. 

Initialization  costs  for  trainers  and  ATE  equipment  not 
currently  supported  by  TICARRS  are  not  included. 

The  report  assumes  that  all  missing  TICARRS  functions  can 
be  provided  with  30,000  additional  lines  of  code  (Page 
VII -30,  Paragraph  C.3.e{l).  Based  upon  current 
CAMS/REMIS  system  sizing,  LCS  believes  the  SLOC  estimate 
is  understated  by  more  than  axi  Order  of  Magnitude  (see 
Attachment  9) .  The  cost  associated  with  a  more  accurate 
estimate  (from  a  source  other  than  DRC)  should  be 
reflected  in  both  the  non-recurring  and  recurring  areas. 

System  Software  Interfaces. 

REMIS  does  not  send  data  to  all  the  systems  listed.  It 
receives  data  from  4  of  the  13  listed.  It  is  easier  to 
develop  an  SSO  than  an  SSI  interface.  The  report  indicates 
that  TICARRS  can  develop  all  4  SSI's  and  9  SSO's  in  1.5  staff 
years.  It  took  .5  staff  years  to  develop  the  K002  interface 
and  that  is  an  SSO.  Based  on  CAMS/REMIS  experience,  it  would 
take  7.3  staff  years  to  develop  these  interfaces. 

PAGE  Vll-23  PARA  D 

None  of  the  TICARRS  documentation  was  developed  lAW  AFR  700 
Series  Regulations  or  DOD-I-7932.  The  cost  to  bring  existing 
documentation  up  to  steuidards  cuid  create  future  documentation 
lAW  standards  is  too  low  at  10%  of  application  software 
investment.  This  figure  would  more  likely  exceed  20-25%  of 
application  software  investment. 

PAGE  VII -24 

The  cost  for  many  weapon  systems  are  not  included.  For 
example,  the  B-2,  C-141,  E-4,  VC-137,  C-22,  C-140,  and  AGE 
equipment . 

PAGE  VII -24,  TABLE  VII -6 

There  are  hundreds  of  different  aircraft  configurations;  most 
are  not  addressed  here.  Some  of  the  obvious  omissions  include 
B2,  C-141  owned/ ope  rated  by  AFMC  (many  different  series)  ,  E-4, 
Special  Air  Mission  Aircraft  (VC-137,  C-22,  C-140,  etc.),  and 
many  -135  and  -130  mission/series  designators. 
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Same  comments  can  be  made  about  missiles  cmd  AGE  equipment. 

PAGE  VII-27,  PARA  3 

User  training  only  accounts  for  instructor  time.  There  will 
be  thouscuids  of  people  (trainees)  taken  off  their  job  for 
TICARRS  training.  Costs  should  be  added. 


The  basis  for  the  eibove- mentioned  cost  analysis  and  its 
conclusions  appear  to  be  seriously  flawed. 

Also,  while  the  IDA  team  considered  only  two  alternatives, 
several  others  are  possible  and  may  have  considerably  more 
merit.  We  have  included  two  additional  Alternatives  (lad>eled 
Alternative  3  and  Alternative  4)  as  Attachments  7  and  8  to 
these  comments. 

The  IDA  report  analysis  concludes  that  there  is  a  cost 
differential  of  $106M  favorable  to  TICARRS. 

Alternative  3  presented  herein  as  Attachment  7,  concludes  that 
there  would  be  cost  differential  of  $147M  favoradale  to 
CAMS/REMIS. 

Alternative  4  presented  herein  as  Attachment  8,  concludes  that 
there  would  be  a  cost  differential  of  $206M  favorable  to 

It  seems  that  more  work  on  the  cost  estimates  is  warranted. 
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Litton _ 

Computer  Services 


<02C  E»ecc«ve  C^.e 
Oavun  Ohio 
4S43C 

(513)  429^i0 


February  3,  1993 


Mr .  Dave  Roberts 

H144  U.S.  Capitol  Building 

Washington/  D.C.  20515-6018 

Dear  Dave, 

It  was  a  pleasure  seeing  you  at  the  Aserican  Xxtgistics 
Infrastructtire  Improvement  Consortium  luncheon  last  Monday. 

After  lunch  when  we  discussed  your  comment  regarding  the 
CAMS/REMIS  Program  "dropping  data,"  you  afforded  me  the 
opportunity  to  respond  to  that  criticism. 

Both  CAMS/REMIS  and  TICARRS  are  chartered  to  provide 
accurate,  timely  operational,  supply  and  maintenance  data 
regarding  the  status  of  individual  end  items  e.g.  airplanes, 
missiles,  automated  test  equipment,  etc.,  and  this  data, 
translated  into  meaningful  information,  plays  an  important 
role  for  the  near-term  deployed  systems  and  the  mid-term 
soon-to-be  deployed  systems.  In  order  to  fairly  co^are  the 
"dropping  of  data,"  or  error  rates,  a  number  of  factors  must 
be  considered.  For  example,  CAMS/REMIS  performs  this 
fvinction  for  significantly  more  end  items  than  TICARRS: 


ISBS 

samg/RB«g 

ZXfiABBfi 

e 

Aircraft 

10,555 

2800 

e 

Missiles 

4,395 

None 

e 

Communications 

Electronics 

1,054,000 

None 

e 

Aerospace  Ground 

60,000 

500 

Equipment  ( including 
Automated  Test  Equipment) 

e 

Simulators  and  Trainers 

1,801 

None 
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:ve  will  Hai.ver  aefoct-free  proOucts  ana  servKBs  on  time  to 


our  internal  end  external  customers 


Utton _ 

Computer  Services 


Nr.  Dave  Roberts 
Page  2 

February  3,  1993 


Because  TICARRs  concerns  itself  alaost  exclusively  with 
aircraft,  and  because  the  Air  Force  personnel  responsible  for 
inputting  data  into  CAMS  for  aircraft  are  such  sore  rigorous 
than  those  responsible  for  less  ■ission^^itical  or  life- 
threatening  itens  such  as  test  equipment,  trainers,  etc.,  a 
fair  conparison  should  restrict  itself  to  aircraft.  The  Air 
Force/Litton  CAMS/REMIS  teas  did  a  study  in  January,  1993  to 
determine  the  error  rate  for  CAMS/REMIS  aircraft  only.  Our 
analysis  of  1,191,903  transactions  showed  64,083  rejects  for 
an  error  rate  of  only  5.38%,  giving  CAMS/REMIS  an  accuracy 
rate  of  94.62%. 

Since  REMIS  and  TICARRS  both  receive  their  data  from  the  same 
soxurce,  CAMS,  any  difference  in  accuracy  rates  would,  almost 
certainly,  result  from  differences  in  editing  of  the  data  by 
REMIS  and  TICARRS  that  is  received  from  CAMS,  e.g.  if  an 
Equipment  Designator  (ED)  such  as  F016F  is  received  by 
TICARRS  it  will  be  accepted  as  valid  even  though  thiure  is  no 
F016F,  and  that's  okay  since  TICARRS  tracks  by  Block  Nxxmber 
and  not  Equipment  Designator.  REMIS,  on  the  other  hand, 
tracks  by  both  Equipment  Designators  and  Block  Numbers  and 
rejects  data  that  is  not  reconcilable  from  one  to  the  other. 
This  is  not  intended  as  criticism  of  the  TICARRS,  but  is  only 
offered  by  way  of  explanation  as  to  why  the  CAMS/REMIS 
accuracy  rate  might  be  slightly  different  from  TICARRS. 

Also,  for  all  of  the  problems  identified  in  the  inputting  of 
data  to  CANS  by  the  functional  users  (in  particular  that  data 
associated  with  items  other  than  aircraft)  corrective  action 
has  already  been  taken,  or  is  underway. 

Thank  you  for  this  opportunity  to  provide  you  with  this 
information. 

Sincerely,  . 

Thomas  F. 

REMIS  Prograun  Direeror 


:b 
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We  ^lldeln^ra,fect.freepn>ducts  •na  eemce,  on  time  to  our  intemel  m^d  extent,  customers 
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Computer  Services 


CAMS/REMIS 


Computer  Services 


CAMS/REMIS 


Computer  Services 


CAMS/REMIS 
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Computer  Services  iTSMte 
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Computer  Services 
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Computer  Services 


CAMS/REMIS 
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Computer  Services 
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loose  visibility  of 
asset  conditions  and 
availability  which 
directly  impacts  mission 
capability. 


CAMS/REMIS 
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Computer  Services 


CAMS/REMIS 
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Computer  Services 


CAMS/REMIS 
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Computer  Services 


CAMS/REMIS 


Computer  Services 
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Computer  Services 
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Computer  Services 
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REMIS  USERS 

(INDIVIDUALS  AUTHORIZED  ACCESS) 


i  mcLiN  I  i  rife  AIR  FORCE 
HKAOQUARTKRS  UNITSO  STATCS  AIR  FORCX 


HQ  USAF/LG 

1030  Air  Forca  Pantagon 
Washington,  DC  20330-1030 


am  1993 


Honorabla  Danial  K.  Znouya 
Chairman,  Subcommittaa  on  Oaf ansa 
Consnittaa  on  Appropriations 
United  Statas  Sanata 
Washington,  D.C.  20S10-602S 

oaar  Mr.  Chairman 

As  I  indicated  to  you  in  my  17  Fab  93  latter,  we  ware 
preparing  to  conduct  an  operational  assessment  of  TZCAHRS  92, 
developed  by  the  Dynamics  Research  Corporation  (DRC),  at  Seymour 
Johnson  APB,  NC.  We  have  completed  the  assessmant  and  z  wish  to 
taka  this  opportunity  to  forward  a  copy  of  the  Assassment  Team's 
report.  I  believe  that  the  assessmant  detailed  In  this  report  was 
both  thorough  and  balanced.  Zn  addition  to  the  Wing  maintenance 
personnel,  wa  had  127  Air  Force  personnel  temporarily  assigned  to 
Seymour  Johnson  AFB  for  11  weeks  to  ensure  we  avaluatad  TZCARRS  92 
in  a  realistic  scenario.  We  spent  an  estimated  $1.3  million  on 
this  effort. 

Let  me  review  the  rules  of  engagement  employed  during  the 
assassment: 

-  TICARBS  92  functionality  was  assessed  on  its  ability,  as 
portrayed  to  the  Air  Force  by  ORC,  to  support  wing  operations 
using  CAMS/REMZS  functions  as  a  baseline. 

-  TZCARRS  92  functional  eapabllltler  were-  baselined  as  or 
11  Feb  93. 

-  CAMS/REMIS  was  kept  current  using  dual  reporting  procedures  to 
maintain  data  base  integrity. 

-  Any  procedures  or  workarounds  identified  to  those  systems  where 
TZCARRS  92  was  non- functional  wore  documented,  made  part  of  the 
test  plan,  and  included  in  the  report. 

-  Software  would  be  baselined  and  frozen  prior  to  the  assessment. 

The  report  details  that  TZCARRS  92  has  positive  features. 
However,  it  lacked  functionality  in  6  of  the  22  CAMS  major 
subsystems:  the  standard  base  supply  system,  comprehensive  engine 
management  system,  personnel/training,  automated  forms,  aerospace 
ground  equipment  management  and  communications-elactronics .  Zn 
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addition,  throa  othar  CAMS  subsystama,  which  tha  contractor  bad  I 
statad  wara  availabla  in  TICARRS  92:  product  quality  deficiency  * 
reporting,  query,  and  cannibalizations  did  not  function.  This 
lack  of  functionality  in  a  total  of  9  out  of  22  subsystems,  ■ 

coupled  with  numerous  other  problems  identified  during  the  ■ 

assasamant,  indicates  that  TICARRS  92  was  clearly  not  ready  to 
operate  as  advertised,  and  in  its  tested  configuration  did  not  ■ 
provide  a  significant  improvement  over  CAMS/Remis.  Reflecting  th| 
above  shortfalls,  in  the  assessed  ability  to  help  perform  duties, 
training  and  overall  satisfaction  with  the  two  systems,  the  usersM 
preferred  CAMS/REMIS.  g 

Based  on  this  report,  we  believe  the  Government  would  incur  . 
significant  costs  to  fully  replicate  the  functionality  currently  I 
resident  in  CAMS/REMIS  and  to  correct  the  TICARRS  92  functional  ■ 
disconnects  Identified  by  the  team.  The  exact  cost  to  develop 
these  features  will  be  provided  by  the  Institute  for  Defense  I 
Analysis  (IDA)  in  their  Congressionally  directed  independent  cost! 
analysis  of  the  two  systems  due  to  be  completed  in  September  93. 

Appendix  0  of  this  report  contains  Assessment  Team  comments  I 
in  response  to  questions  from  the  Air  Force  Program  Executive 
Officer  for  Information  Systems.  Comments  provided  to  the  Air  e 
Force  in  a  recent  DRC  dooiment  presented  quotes  from  these  | 

responses;  those  quotes  are  included  in  their  full  context  in  the 
report.  I  think  you  will  find  the  complete  quotes  of  more  value  . 
in  assessing  TICARRS  92.  I 

The  next  step  is  the  completion  of  the  IDA  study.  We  look 
forward  to  receiving  that  report  and  working  with  you  on  final  I 
resolution  of  this  issue,  we  plan  to  make  no  ;3udgements  relative  * 
to  the  future  of  TICARRS  92-CAMS/REMIS  until  we  have  thoroughly 
reviewed  the  IDA  study.  We  will  then  provide  you  final  1 

recommendations  on  this  issue.  In  the  meantime,  TICARRS  will  I 
continue  to  operate  as  before  at  those  units  it  currently 
supports.  Additionally,  in  the  interest  of  maintaining  the  most  I 
combat,  ready  forces  peesible,  r  intend  to  continue  with  the  planned! 
CAMS/REMIS  enhancwnents  to  better  serve  our  users,  and  provide  the 
best  possible  maintenance  management  information  system.  j 

I  appreciate  your  continued  interest  and  support  of  this 
effort,  and  l  will  keep  you  advised  of  any  future  developments.  . 


Sincerely, 


'■^revor  A.  Hammond,  Lt  Gen, 
DCS/Logisties 


USAF 
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COURSE  NO: 

EIOSH4925-011 

COURSE  NAME:  REHISTALK 

USERS  COURSE 

YOUR  NAME: 

DATE: 

JOB  TITLE: 

PHONE: 

DEPARTMENT: 

BASE: 

Please  cirde  the  response  that  most  closely  reflects  your  impression  of  this  training,  insttucmr. 
the  materials,  and  the  content  that  was  presented.  Your  feedback  helps  improve  the  quaiiiy 
of  the  trainmg  that  is  ottered. 

1.  INSTRUCTOR 

EXCELLENT 

UNACCEPTABLE 

a)  overall  effectiveness 

6 

5 

4  3  2  1 

b)  knowiedge  of  sut^  matter 

6 

5 

4  3  2  1 

c)  held  your  interest 

6 

5 

4  3  2  1 

d)  amount  of  interaction/questions 

6 

5 

4  3  2  1 

e)  organization 

6 

5 

4  3  2  1 

f)  met  the  course  objectives 

6 

5 

4  3  2  1 

Quality  of  Instruction  Comments: 


II.  WRITTEN  MATERIALS  (EXCEPT  CAI) 

EXCELLENT 

UNACCEPTABLE 

a)  organization  of  materials 

6 

5 

4 

3 

^  1 

b)  ease  of  understanding  the  materials 

6 

5 

4 

3 

2  1 

c)  quality  of  printed  materials 

6 

5 

4 

3 

2  1 

d)  quality  of  view  graphsArisual  aids 

6 

5 

4 

3 

2  1 

e)  appropriateness  of  examples 

6 

5 

4 

3 

2  1 

f)  educational  value  of  the  exercises 

6 

5 

4 

3 

2  1 

Qualitv  of  Written  Material  Commenic:- 

ATTACHMENT  5 

IV-69 


17377 


COURSE  NO:g^Qg^^g25.009 

COURSE  NAME: 

REMIS  GCSAS 

USERS  COURSE 

your  NAME: 

DATE: 

JOB  TITLE: 

PHONE: 

DEPARTMENT: 

BASE: 

Please  circle  the  response  that  most  closely  reflects  your  impiesston  <4  IMs  ooiffse,  instnjcMr, 
the  material,  and  the  content  that  was  presented.  The  CAl  Is  rated  on  pages  3  ttuu  6.  Your 
feedback  will  help  improve  the  quality  of  ttw  training  that  is  offered. 

1.  INSTRUCTOR  EXCELLENT  UNACCEPTABLE 

a)  overall  effectiveness 

6 

5  4 

3 

2  1 

b)  knowledge  of  subject  matter 

6 

5  4 

3 

2  1 

c)  held  your  interest 

6 

5  4 

3 

2  1 

d)  amount  of  interaction/questions 

6 

5  4 

3 

2  1 

e)  organization 

6 

5  4 

3 

2  1 

f)  met  the  course  objectives 

ni.ali»u  Inetnir^inn  rVimmAntc 

6 

5  4 

3 

2  1 

11.  WRITTEN  MATERIALS  (EXCEPT  CAi)  EXCELLENT 

UNACCEPTABLE 

a)  organization  of  materials 

6 

5  4 

3 

2  1 

b)  ease  of  understanding  the  materials 

6 

S  4 

3 

2  1 

c)  quality  of  printed  materials 

6 

5  4 

3 

2  1 

d)  quality  of  view  graphs/visuai  aids 

6 

5  4 

3 

2  1 

e)  appropriateness  of  examples 

6 

5  4 

3 

2  1 

f)  educational  value  of  the  exercises 

6 

5  4 

3 

2  1 

Oualitv  of  Written  Material  Comments: 

y  . 
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ALTBSNATIVB  3 


CONSOLIDATSD  CAICS/SSKZS  OPERATION 

CAMS/REMIS  is  consolidated  and  modified  and  remains  the  Air  Force 
standard  system.  TICARRS  is  retired. 

This  Alternative  provides  for  data  editing  and  control  and  other 
CAMS  enhcuicements  outlined  in  Alternative  I.  In  addition,  it  would 
move  the  CAMS  operation  to  the  REMIS  facility  amd  provide  for  a 
computer  to  operate  CAMS  like  TICARRS  from  a  central  location. 
User  support  would  be  provided  as  in  REMIS. 

As  a  result  of  a  consolidated  CAMS/REMIS  operation,  the  cost  of 
CAMS  Data  Base  Managers  ($54M)  would  be  eliminated.  The  RPC  costs 
of  $186  million  would  be  decreased  to  the  cost  of  am  additional 
computer  from  the  CAMS  portion  of  CAMS/REMIS. 

With  a  CO- located  operation,  the  cost  of  the  high-speed  REMIS  data 
links  would  be  eliminated. 

Chart  I  shows  costs  for  a  co- located  CAMS/REMIS  operation. 
Additional  benefits  by  operating  CAMS  like  REMIS  would  be  the 
increased  availeibility  equal  to  REMIS  amd  a  system  response  time, 
amoimting  to  under  3  seconds. 

1.  Non-RecTirring  Costs 

a .  Hardware 

New  hardware  non-recurring  costs  are  projected  at  $13M  to 
provide  a  centralized  CAMS  operation.  Some  of  this  cost 
could  be  greatly  reduced  with  the  utilization  of  excess 
stamdard  base  level  cooputers  which  are  removed  under  I^4R 
924. 

Year  seven  hardware  replacement  costs  are  determined  by 
using  IDA'S  90%  replacement  figures  for  equipment 
replaced  after  seven  years.  Technology  advances  over  the 
next  seven  years  should  allow  for  the  purchase  of  the 
same  performance  at  a  cost  of  $3.4M  (assuming  the  20% 
compotmd  annual  cost  reduction  for  mainframe  computers 
equipment  continues) . 

b.  Software 

This  element  of  the  cost  estimate  is  concerned  with  the 
cost  of  further  development  of  the  CAMS  application  after 
1993.  The  CAMS/REMIS  PMO  is  not  projecting  any  funds  for 
this  purpose  past  1993,  except  those  funds  provided  by 
organizations  wishing  to  enhcmce  CAMS  to  support  a 
specific  weapon  system  or  activity. 
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J^qpllcation  Software 

Approved  fuactioxial  modifications 

CAMS  has  approved  fiinctional  modifications  (CSRDs) 
amounting  to  approximately  30  person-years  of  effort 
(CAMS  estimate)  .  This  is  projected  to  extend  over  a 
three  year  period  or  10  person-years  per  year  for  three 
years . 

Enhanced  data  editing  and  control 

CAMS  must  be  modified  so  that  the  data  entry  editing  is 
not  only  less  onerous,  but  fully  edited,  such  that 
erroneous  data  is  refused  at  the  point  of  entry,  and  at 
the  time  of  entry.  This  enhancement  would  totally 
replace  the  current  process  of  REMIS  responding  to  the 
individual  bases  with  error  messages  an  hour  or  two  after 
the  fact,  with  the  expectation  of  error  correction  by  the 
flight  line  personnel.  The  estimates  for  enhanced  data 
editing  and  control  would  require  about  15  person-years 
of  development  team  effort  per  year  for  three  years, 
including  the  ability  to  communicate  immediately  with 
REMIS  for  data  verification  on  a  fleet -wide  basis  as 
necessary. 

CAMS  Extensions  and  new  weapon  support 

Funding  is  included  to  facilitate  the  addition  of 
forthcoming  requests  for  CAMS  support  for  the  F-22,  IMIS, 
REMPOD,  Job  Cost  Accounting,  CEMS  interface,  deployable 
CAMS,  and  REMIS  support.  This  funding  is  10  person-years 
for  three  years. 

All  of  the  work  includes  documentation  and  is  projected 
to  be  completed  by  contractor  personnel  at  the  rate  of 
$60  per  hour. 

Enhanced  Cl lent /Server 

Under  this  alternative,  the  centralized  CAMS  requires 
additional  software  to  support  directing  data  to  the 
correct  logical  CAMS  data  base.  This  software  is 
estimated  at  22,300  SLOC. 

Report  page  VII-30  specifies  30,000  SLOC  as  the  size  of 
the  8  functional  enhancements. 

Report  page  VII -i  specifies  a  staff  year  as  equal  to 
1,920  hours. 

Report  page  Vll-2  specifies  that  software  engineering 
Icdaor  is  equal  to  $60/hour  or  $115 , 200/staff  year. 
Report  page  VII- 21  specifies  that  the  eight  enhancements 
will  require  14  staff  years  to  develop. 
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Based  upon  these  references,  SLOC  per  staff  hour  was  calculated  as 
follows : 

14  staff  yrs  x  1,920  hours  per  yr  *  26,880  staff  hours 

30,000  SLOC  divided  by  26,880  staff  hrs  =  1.1160714  SLOC  per  staff 

hour 

1.1160714  (prod  rate)  x  22,300  SLOC  *  24,888  development  hours 

$60/hr  rate  x  24,888  dev  hours  =  $1,493,280 


System  Integration  and  Test 

This  function  is  currently  performed  by  the  SSC  QT&E 
group  at  Gunter  AFB.  The  costs  projected  for  this  item 
are  a  function  of  the  application  development  effort. 
The  study  team  estimated  that  15  person-years  per  year 
for  the  first  three  years  would  cover  the  necessary 
effort.  This  not  only  includes  integration  of  the  CAMS 
application  into  the  SBLC  and  RPC  software,  but  also 
includes  distribution  to  the  various  using  organizations. 
This  work  is  projected  at  an  annual  rate  of  $43,099  per 
staff  year. 

2 .  Recurring  Costs 

a.  Goverxment  Program  Management 

This  is  the  projected  expense  for  program  management  of 
C7^.  This  cost  is  estimated  to  be  10%  of  the  staff 
supporting  CAMS  in  development,  maintenance,  and  bases 
representatives . 

b.  Computer  operations  and  maintenance 

Operation  of  a  centralized  operation  would  require  twelve 
people  in  this  category.  The  annual  rate  of  $44,576  x  12 
is  what  IDA  has  estimated  for  this  area. 

Hardware/ software  operation  costs  are  based  on  a 
percentage  of  IDA's  costs  in  the  amount  of  $3.59M  for 
five  RPCs  for  CONUS  operations.  The  cost  of  CAMS  has 
been  estimated  by  IDA  to  rise  40%  per  RPC,  this 
percentage  was  used  as  the  factor. 


c.  Base  communication  operations 

This  cost  was  devised  for  support  of  one  E-7  @  107  base 
for  communication  equipment  maintenance  and  terminal 
support.  The  projected  E-7  cost  is  $49,591  per  year  per 
representative,  as  per  the  IDA  study. 

Costs  for  hardware  and  software  are  based  on  Commercial 
off  the  shelf  (COTS)  client/server  operation  and 
equipment . 


d.  User  Support 

User  support  will  be  provided  in  the  seune  manner  as 
currently  provided  by  REMIS.  User  support  personnel 
costs  are  based  on  IDA  $41,299  per  person-year.  Twelve 
personnel  will  provide  user  help  at  a  consolidated 
function. 

e.  Software  Maintenance 

Application  Maintenance.  This  cost  element  provides 
funds  for  the  maintenance  of  CAMS  software.  Software 
maintenance  includes  correcting  and  testing  software 
er^'ors  reported  by  users  (Discrepancy  Reports  or  DIREPs)  , 
making  modifications  to  the  CAMS  data  base  to  enhance 
performance,  or  add/ccrrect  data  relationships.  Recent 
CAMS  history  indicates  chat  about  40%  of  the  CAMS  staff 
(45  staff  members)  are  needed  to  address  the  current 
maintenance  workload.  The  study  team  estimates  that  this 
level  of  effort  will  be  required  for  the  foreseeadile 
future.  Since  CAMS  consists  of  approximately  l.l  million 
lines  of  unique  source  code,  this  results  in  the 
maintenance  of  24,961  unique  lines  of  source  code  per 
staff  year.  This  is  in  line  with  typical  industry 
averages  for  data  base  applications  which  vary  between 
20,000  and  32,000  unique  lines  of  source  code  per  staff 
year  (Boehm,  p.  541] .  This  work  will  be  performed  by  a 
mixture  of  military  and  civilian  personnel  at  an  average 
rate  of  $43,099  per  year.  This  line  item  includes  funds 
for  computer  support  and  system  integration  and  release 
as  follows: 


Operations.  This  item  projects  the  cost  of  providing  the 
necessary  computer  support  to  the  CAMS  application 
maintenance  personnel.  The  cost  detail  is  described  in 
Tcible  A.  The  maintenance  effort  is  projected  to  require 
two  computer  systems  to  perform  its  work. 


Table  A  CAMS  development  operation  support  estimates 


Hardware  Maintenance 
Software  Maintenance 
Computer  operators  (5) 
Supplies,  Misc. 
Subtotal 


$  71,000  per  system  per  year 
$  20,000  per  system  per  year 
$150,000  per  system  per  year 
$  20,000  per  system  per  year 
$260,000  per  system  per  year 


System  Integration  and  Test.  This  function  is  performed 
by  the  QT&E  group  at  the  Gunter  AFB  SSC.  The  cost 
projections  are  based  on  a  prorated  share  of  the  current 
QTSeE  support  (75  personnel)  .  About  half  of  the  current 
personnel  are  allocated  to  CAMS.  CAMS  maintenance  is 
expected  to  drop  to  edDout  40%  of  its  current  level.  We 
estimated  40%  of  the  37  QT&E  personnel  to  be  required  for 
maintenance  support  at  a  cost  of  $43,099  per  person-year. 
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Data  Base  Management.  This  effort  is  staffed  by  the  CAMS 
organization  at  the  Gunter  SSC  and  includes  making 
modifications  to  the  CAMS  data  base  to  enhance 
performance,  or  add/correct  data  relationships  that  are 
encountered  by  the  users.  The  study  team  estimated  this 
effort  to  require  four  experienced  data  base  managers  at 
an  average  rate  of  $43,099  per  year.  This  function  would 
move  to  the  central  CAMS/REMIS  operation. 

Documentation.  These  funds  include  resources  to  modify 
and  update  the  software  documentation  in  accordance  with 
the  changes  due  to  the  software  and  data  base 
maintenance.  This  is  estimated  to  require  two  staff 
years  per  year,  which  is  5%  of  the  application  software 
resources . 

Training  and  Travel.  Training  costs  of  the  CAMS  users 
are  carried  by  the  individual  bases  and  are  expected  to 
be  substantially  the  same  regardless  of  the  specific 
weapons  maintenance  information  system  used.  The  CAMS 
base  representatives  will  require  training  on  a 
continuous  basis  as  personnel  are  reassigned.  The 
training  and  travel  costs  are  estimated  to  be  2%  of  the 
CAMS  controlled  funds. 

New  Recurring  Cost.  Sprint  communication  lines  as 
prepared  for  TICARRS  would  be  used  for  CAMS.  This  would 
be  107  dedicated  lines  to  a  centralized  CAMS  operation. 
Costs  are  based  on  IDA  costing  frr  TICARRS. 

Lines  to  RPC.  This  cost  is  based  on  the  IDA  cost  per 
Table  VII -4  2.4.2  for  TICARRS  RPC  line  costs. 

Data  Base  System  Manager.  Data  base  management  and 
functional  support  in  a  consolidated  operation.  A  total 
of  ten  data  base  managers  and  five  functional  personnel 
will  be  required  for  this  aspect  of  the  operation. 

Cost  for  the  DBM  are  based  on  IDA  DBM  cost  of  $43,000. 
Function  support  are  based  on  REMIS  costs  of  $36,000  per 
person  per  year. 
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PROJECTED  MODIFIED  CANS  COSTS  FOR  YEARS 
1994  -  2003  (MILLIONS  IN  1994  DOLLARS) 
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TOTAL  COST  **31.755  **20,685  **20.685  **14.667  **14,667  **14.667  **14.678  **18.067  **14.667  **14.667  **  179.205 


CHM?  2 
AITHNATIVE  3 
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ALTERKATIVE  4 

CONSOLIDATED  CAMS/REMIS  WITH  REMI8  HARDWARE  UPGRADE 

CAMS/REMIS  is  consolidated  and  modified,  the  REMIS  hardware  suite 
is  upgraded,  and  CAMS/REMIS  remains  the  Air  Force  standard  system. 
TICARRS  is  retired. 

This  alternative  approach  builds  on  Alternative  3,  by  including  all 
initiatives  in  Alternative  3  and  adding  a  REMIS  hardware  upgrade  of 
the  current  Tandem  VLX  suite  to  Tandem  Cyclones.  In  addition, 
currant  tape  back  up  and  archiving  would  be  replace  with  optical 
disk  technology.  Further  current  systems  operating  at  the  five 
AFMC  ALCs  would  be  migrated  to  Cyclone  processors. 

Costs  associated  with  this  alternative  have  been  well  documented  in 
a  Business  Case  developed  by  the  REMIS  PMO.  An  investment  of 
approximately  $19  M  in  replacement  Tandem  equipment  will  net  a 
continuous  savings  in  excess  of  $8  M  per  year,  every  year,  for  the 
life  of  AFMC  Tandem  operations.  This  equates  to  a  pay  back  on 
investment  of  under  three  years.  Chart  1  portrays  these  savings, 
and  identifies  a  further  gap  in  ten  year  cost  projections  between 
the  CAMS/REMIS  and  TICARRS  options. 

1 .  Mon-Reeurr Ing  Costs 

a.  Hardware  Purchase  and  Maintenance 

Projected  first  year  hardware  upgrade  costs  are  estimated 
to  be  $16.8  M.  Included  in  these  costs  are  the 
replacement  of  all  REMIS  Tandem  VLX  hardware  with  Tandem 
Cyclones.  Cyclone-R  equipment  would  be  installed  at  the 
five  ALCs,  replacing  all  TXP,  NSII,  and  VLX  equipment 
currently  in  place.  Current  procedures  for  tape  back  up 
and  archiving  would  be  replace  through  use  of  newly 
installed  optical  disk  storage  devices  at  the  five  ALCs 
and  at  Headquarters  AFMC.  The  current  REMIS  Network 
Control  Center  (NCC)  would  be  upgraded  to  accommodate  the 
increasedrequirements  of  operating  additional  ALC  systems 
(Exchangeable  Production  System  -  EPS,  Inventory  Tracking 
System  -  ITS,  and  the  Depot  Time  and  Accounting  System)  . 
In  addition,  costs  have  been  included  for  an 
uninterrupted  power  source  (UPS) ,  to  ensure  continuous 
service  even  in  power  outage  situations.  Hardware 
installation  is  also  included  in  this  projection.  First 
year  maintenance  is  included  in  the  purchase  price  of  the 
upgrade.  Thereafter,  system  maintenance  is  vastly 
reduced  since  a  single  configuration  will  be  resident  at 
the  ALCs  and  the  number  of  processors  required  to  operate 
REMIS  under  Cyclone  technology  will  be  reduce. 

b.  Software 

Included  in  the  cost  of  hardware  installation,  is  the 
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conversion  of  all  current  system  software  to  the  Tandem 
C-30  release.  In  addition,  all  software  not  now 
operating  of  Cobol  85  will  be  upgraded  to  that 
capability. 


2 .  Recurring  costs 

a.  Personnel  Costs 

As  a  result  of  the  hardware  system  upgrades,  personnel 
operating  costs  are  drastically  reduced.  Analysis  shows 
there  will  be  a  net  savings  of  65  personnel  equivalents 
based  on  this  alternative.  Total  recurring  personnel 
costs  will  be  reduced  from  $7.8  M  to  $2.1  M  per  year. 
This  is  possible  through  use  of  lights  out  technology. 
Since  all  operations  of  ALC  Tandem  equipment  will  be 
controlled  from  the  REMIS  NCC,  at  Headquarters  AFMC,  no 
operations  personnel  are  required  at  the  ALCs.  Personnel 
support  required  under  the  upgrade  is  limited  to  staff 
increases  at  Headquarters  AFMC  REMIS  operations.  This 
will  accommodate  an  increase  in  customer  support 
requirements  from  existing  ALC  systems  and  for  existing 
system  programmer  support.  In  addition,  costs  have  been 
included  for  a  performance  and  capacity  planning  team. 
Experience  has  shown  that  as  applications  mature  and 
change,  there  is  a  continuous  need  to  review  the 
methodology  used  in  employing  the  use  of  Tandem 
technology.  This  constant  review  allows  for  the  most 
efficient  operations  and  lowest  cost  per  processing 
second.  With  the  requirement  to  support  three  additional 
systems,  there  will  be  a  significant  cost  benefit  to 
increase  the  current  team  to  monitor  these  additional 
systems . 

b.  Facility  costs 

Environmentally  controlled  space  requirements  are 
significantly  reduced  by  implementing  this  alternative. 
Current  system  configurations  require  in  excess  of  4000 
square  feet  of  footprint.  The  proposed  upgrade  would 
reduce  this  requirement  to  just  over  600  square  feet. 
Thus,  yearly  heat,  power,  and  footprint  costs  are  reduced 
by  $700  K  per  year. 


3.  Existing  System  Cost 

a.  Hardware  Maintenance 

Maintenance  of  all  AFMC  system  is  a  significant  yearly 
cost.  Under  current  configurations,  AFMC  is  incurring  a 
yearly  maintenance  cost  for  Tandem  equipment  in  excess  of 
$1.5  M. 
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b.  Parsonn*!  Costs 

System  support  costs  for  ALC  personnel  are  a  major 
element  of  existing  systems  operating  costs.  Personnel 
support  for  EPS,  ITS,  and  the  Time  and  Accounting  System, 
amount  to  over  $7.8  M  per  year. 

c.  Facility  Costs 

Heat,  power  and  footprint  costs  to  operate  current  Tandem 
configurations  at  all  locations  are  in  excess  of  $800  K 
per  year. 

d.  Existing  System  Cost  Summary 

Total  system  support  costs  for  all  AFMC  Tandem  equipment 
amount  to  nearly  $10.5  M  per  year.  The  data  portrayed  in 
this  section  is  fully  substantiated  in  the  MSC/SRB 
Business  Case  For  The  Consolidation  and  Upgrade  of  AFMC 
Tandem  Computers,  dated  22  June  1993. 


4.  Mat  Cost  Summary 

In  order  to  implement  this  alternative  there  is  an  initial 
investment  of  approximately  $19  M.  This  however,  is  offset  by 
a  continuous  yearly  savings  in  excess  of  $8  M.  Thus  after 
just  over  two  years,  there  is  a  complete  return  on  investment 
and  a  further  continuous  savings  of  $8  M  per  year.  These 
statistics  have  been  overlaid  on  the  Alternative  3  Chart  l  and 
are  shown  on  Chart  1  of  this  alternative. 
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COST  TO  UPGRADE  TICARRS 
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IV.  DYNAMICS  RESEARCH  CORPORATION 


A.  FORWARDING  LETTER 

A  copy  of  the  forwarding  letter  from  Albert  Rand,  President  and  CEO,  Dynamics 
Research  Corporation  (DRC),  appears  on  pages  V-2  through  V-5  of  this  document. 
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DYNAA^ 

RESEARCH 

CORPORATION 

60  FRONTAGE  ROAD,  ANDOVER.  MASSACHUSETTS  01 81 0  TEL.  506/475-9090 


July  23,  1993 


Mr.  Stanley  A.  Horowitz 
Institute  for  Defense  Analyses 
1801  N.  Beauregard  Street 
Alexandria,  Virginia  22311-1772 

Dear  Mr.  Horowitz: 

I  appreciate  the  opportunity  you  provided  to  review  the  draft 
version,  "A  Comparison  of  Air  Force  Data  Systems."  Your  team  did  an 
excellent  job  in  synthesizing  a  vast  eunount  of  information  in  a 
relatively  short  time  on  a  very  complex  subject. 

In  providing  a  copy  of  the  report  to  us,  Mr.  David  Dore  requested 
that  we  review  the  repoirt  for  completeness,  correctness,  and 
accuracy.  Enclosed  you  will  find  the  comments  that  we  believe  to  bs 
substantive  to  an  objective  assessment  of  the  information  contained 
in  the  report.  These  should  not  be  interpreted  as  being  inclusive 
of  all  our  comments  or  critical  of  your  efforts. 

For  convenience,  we  have  organized  our  comments  into  two 
categories.  All  of  our  comments  relating  to  cost  are  attached  to  a 
cost  element  spread  sheet  depicting  a  comparison  of  CAMS/REMIS,  and 
TICARRS  costs  (Tab  A).  In  some  cases,  these  comments  also  relate  to 
the  narrative  discussion  in  the  report.  The  second  category  of 
comments  relates  to  the  contents  of  the  remainder  of  the  report 
(Tab  B) .  Our  comments  relate  primarily  to  errors  of  fact,  cases  of 
incomplete  information,  and  omissions  which  materially  impact  the 
conclusions  and  create  a  negative  impression  inconsistent  with  the 
actual  facts .  Our  objective  was  not  to  critique  every  word  or  the 
style  in  which  the  report  was  written.  Tab  C  is  a  copy  of  the  MOA 
for  the  Operational  Assessment. 

Operational  Assessment 

My  personal  review  of  the  report  did  raise  several  issues 
concerning  the  methodology  used  in  the  study  and  the  logic  of  the 
arguments  presented  that  are  of  concern  to  me.  I  would  like  to 
share  these  with  you. 

My  first  concern  relates  to  the  Operational  Assessment  data. 
Contrary  to  popular  belief  this  was  not  a  6  week  test  and  it  was 
far  from  being  an  objective  assessment.  The  conclusions  in  the  Air 
Force  report  are  inconsistent  with  the  Air  Force's  own  experience 
where  over  2,350  F-15,  F-16,  and  F-117  aircraft  flying  over 
6,283,200  hours  have  been  successfully  supported  by  TICARRS.  We 
have  reviewed  the  Air  Force  assessment  report  and  we  have 
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identified  and  sent  a  letter  to  the  Air  Force  identifying  a  few  of 
the  major  errors  of  fact  which  have  a  significant  bearing  on  an 
objective  assessment  of  that  test.  From  reading  the  IDA  report,  it 
appears  that  IDA  may  have  carried  forward  a  number  of  these  errors 
of  fact.  Before  we  can  agree  on  the  facts  of  your  report,  we  need 
to  reach  agreement  as  to  the  relevant  facts  of  the  Assessment.  It 
is  my  hope  that  at  the  July  28  meeting  we  can  resolve  these  issues. 

Increasing  the  Scope  of  TICARRS 

A  number  of  caveats  appear  throughout  the  report  that  express 
doubts  as  to  our  and  TICARRS  ability  to  absorb  the  scope  of  the 
inventory  supported  by  CAMS  in  a  timely  fashion  and  retain  high 
data  quality.  I  do  not  agree  with  this  subjective  assessment  based 
on  history.  The  concurrency  of  the  world  wide  F-16  activation 
schedule  and  the  multiple  aircraft  configurations  supported  remain 
unprecedented  in  DOD.  DRC  demonstrated  the  ability  to  rapidly 
activate  TICARRS,  load  and  validate  data  bases,  make  software 
modifications,  sustain  operations,  and  deal  with  a  dynamic  and 
changing  schedule  -  successfully  meeting  contract  requirements  at 
a  fixed  price.  Regarding  data  quality,  as  observed  by  IDA,  the 
design  of  TICARRS  inherently  provides  software  and  procedural 
routines  and  edits  that  insure  high  data  quality.  This  capability 
is  not  sensitive  to  inventory  size.  Certainly,  Air  Force  wide 
activation  of  TICARRS  will  be  challenging,  but  we  have  already  met 
the  challenge  with  three  of  the  most  complex  weapons  systems  and 
associated  logistics  environments  in  DOD.  There  is  no  reason  to 
believe  DRC  and  TICARRS  are  not  up  to  the  challenge. 

Methodology 

A  functional  comparison  of  CAMS/REMIS  to  the  TICARRS-92  baseline 
(direct  connectivity  and  data  base)  would  have  highlighted  a 
significant  and  critical  difference  which  distinguishes  TICARRS 
from  CAMS/REMIS  today  and  for  the  future.  TICARRS  was  designed 
from  the  outset  to  be  an  integrated’,  on-line  weapons  system 
management  information  system  tying  together  all  users  (base, 
depot,  headquarters,  program  offices,  and  contractors)  with  access 
to  timely  and  accurate  information  to  manage  the  logistics  and 
engineering  support  of  complex  weapon  systems .  Although  CAMS 
representj'  an  advance  over  its  predecessor  MMICS,  CAMS  was  designed 
as  a  data  collection  system  with  each  base  being  a  stove  pipe  of 
local  data.  REMIS  was  designed  as  a  data  collector  being  fed  from 
various  other  data  systems .  The  operational  assessment  at  Seymour- 
Johnson  clearly  did  not  demonstrate  the  full  capabilities  of  an 
integrated,  real  time  fleet  wide  information  system  capability. 

TICARRS  may  have  been  ahead  of  its  time  because  it  was  designed  to 
support  a  proactive  management  by  weapon  system  management 
philosophy,  with  fleet-wide  and  asset-wide  visibility.  This 
management  strategy  is  now  being  espoused  in  various  DOD  policies . 
In  this  respect,  TICARRS  represents  the  most  advanced  and  unique 
capability  of  its  type  within  DOD  and  represents  the  most  capable. 
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advanced  capability  to  serve  as  baseline  for  the  incremental 
development  of  the  next  generation  of  weapons  system  information 
systems . 

Data  Quality 

Additionally,  this  approach  and  the  associated  costing  methodology 
do  not  assess  the  cost  and  the  design  issues  associated  with  CAMS 
and  REMIS  meeting  or  exceeding  the  data  quality  capabilities  of 
TICARRS.  The  report  notes  that  TICARRS  does  not  have  a  requirement 
for  data  accuracy  and  validity,  but  that  CAMS  and  REMIS  do  thereby 
creating  the  impression  that  this  is  a  deficiency  in  TICARRS.  These 
requirements  only  address  computational  precision  and  not  validity 
in  terms  of  measuring  the  quality  of  the  data  in  the  data  base . 
These  requirements  are  not  a  measure  of  what  is  missing  or  the 
accuracy  of  what  is  there. 

It  matters  little  how  many  digits  to  the  right  of  the  decimal  point 
the  algorithm  will  calculate  if  the  data  is  unreliable  to  begin 
with.  This  is  not  an  insignificant  issue  because  TICARRS  data  was 
considered  so  complete  and  accurate  that  contractors  accepted  it 
for  warranty  determination  purposes.  General  Dynamics  considered 
TICARRS  data  to  be  so  timely  and  accurate  that  they  tied  TICARRS 
data  as  an  input  to  the  GD  Quality  Assurance  Report  process.  This 
program  called  Project  Bluestreak  was  extremely  successful  in 
correcting  manufacturing  defects. 

Cost 


The  question  of  cost  raises  a  question  of  logic  in  the  analysis. 
Your  report  correctly  noted  the  adequacy  of  the  TICARRS  cost  data 
base  as  well  as  noting  the  uncertainty  of  the  supporting  CAMS 
estimate.  Unfortunately,  the  report  treats  each  point  estimate  with 
the  same  degree  of  certainty  even  though  the  report  states  "past 
costs  was  used  to  assess  the  realism  of  estimates  of  future  costs . " 
The  presentation  of  cost  presents  a  misleading  picture  of  the  risks 
associated  with  each  alternative.  DRC's  contract  history 
demonstrates  our  ability  to  consistently  deliver  on  time  and  on 
budget.  I  do  not  believe  the  record  supports  such  perfoimiance  with 
CAMS  and  REMIS.  For  completeness,  I  believe  the  report  should 
include  in  its  discussion  the  track  of  prior  year  costs,  the  track 
of  estimates  versus  actuals,  and  the  lower  risk  inherent  in  TICARRS 
estimates . 

Further,  the  cost  estimate  penalizes  the  TICARRS  alternative  with 
the  nonrecurring  cost  to  clean-up  the  CAMS  and  REMIS  data  bases  to 
TICARRS  quality  levels.  Clearly,  there  is  a  need  to  clean-up  the 
CAMS/REMIS  data  base  before  conversion  to  achieve  the  benefits  of 
high  quality  data.  This  cost  should  be  a  charge  to  CAMS/REMIS,  not 
TICARRS . 
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TICARRS/CAMS  Transition 


One  additional  issue  is  of  concern.  IDA  attempted  to  test  the 
hypotheses  that  weapon  system  management  information  management 
systems  affect  the  operational  and  logistics  support  environment 
and  that  a  specific  change  from  one  information  system  to  another 
could  be  linked  to  various  logistics  indicators. 

Based  on  the  limited  data  provided  in  the  report,  I  do  not  believe 
the  assertion  that  the  conversion  from  TICARRS  tc  CAMS  did  not 
impact  weapon  system  performance.  For  this  conclusion  to  be  true, 
we  have  to  believe  that  the  significant  and  documented 
deterioration  in  the  quality  of  the  CAMS/REMIS  data  base  upon 
conversion  had  no  impacts  in  the  logistics  community.  However,  we 
know  from  the  large  number  of  testimonial  messages  from  the  F-16 
logistics  managers  and  the  contractors  that  it  had  a  significant 
impact.  Additionally,  bad  actor  studies  by  RAND  and  DRC  have 
demonstrated  the  significant  payoff  resulting  from  high  quality 
d^ta.  Further,  this  conclusion  runs  counter  to  IDA's  rationale  for 
a  data  system  that  provides  over  90-100%  accuracy.  If  quality  data 
were  not  a  significant  cost  driver  impacting  weapon  system 
performance,  the  Air  Force  would  not  have  initiated  the  RIPIT 
project  to  improve  the  quality  of  data.  The  gross  metrics  selected 
for  hypothesis  testing  are  driven  by  a  large  number  of  diverse 
variables  and  to  suggest  that  MKH/FH  and  MTBF  should  be  the  sole 
indicators  of  the  impact  of  the  conversion  to  CAMS  oversimplifies 
the  issue.  I  am  not  aware  of  any  data  that  has  demonstrated  a 
direct  causal  relationship  of  MMH/FH  and  MTBF  as  being  the  only 
determinants  of  weapon  system  performance.  I  suggest  you  delete 
this  discussion  from  the  report. 

We  appreciate  the  opportunity  afforded  us  to  comment  on  your 
excellent  study,  and  we  look  forward  to  the  meeting  on  Wednesday, 
July  28th. 


Sincerely, 

DYNAMICS  RESEARCH  CORPORATION 


Albert  Rand 
President  and  CEO 
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B  .  COMMENTS  ON  COSTS 

DRC  Comments: 

A  copy  of  the  comments  from  DRC  on  costs  (referred  to  as  “Tab  A”  in  the 
forwarding  letter)  appears  beginning  on  page  V-8  of  this  document.  The  page  numbers  in 
both  the  DRC  comments  and  the  IDA  responses  refer  to  page  numbers  in  the  review 
version  of  IDA  Paper  P-2863. 

IDA  Response: 

1 .  Page  VII- 1,  paragraph  6;  We  applied  the  1,920  hours  equally  to  all  systems  and 
used  generic  labor  rates  to  protect  proprietary  information. 

2 .  Page  VII-2,  Section  A,  Cost  Estimate  for  CAMS:  Noted. 

3 .  Page  Vn-5,  Section  Alb,  paragraph  4:  We  believe  current  technology  will  support 
the  interface  and  that  we  have  provided  sufficient  resources  for  it. 

4.  Page  Vn-8,  Section  A2e,  paragraph  4:  We  have  revised  our  cost  estimates  to 
include  6  data  base  maintenance  people  and  4  data  base  management  people  at 
Gunter  SSC.  Costs  for  data  base  managers  at  base  level  are  included  under  2.3.2, 
Base  Reps.  Assistance  provided  by  UNISYS  personnel  at  SSC  is  included  under 
contractor  labor. 

5.  Page  VII-16,  Section  C,  paragraph  1:  We  have  revised  our  schedule  estimates  in 
light  of  the  MAISRC  requirement,  and  our  best  estimate  for  implementing 
nCARRS  is  four  years. 

6.  Page  VII-19,  Section  Clb,  paragraph  1:  Noted.  We  stand  by  our  estimate. 

7 .  Page  VII-21,  Section  Clb,  paragraph  8:  Noted.  We  stand  by  our  estimate. 

8.  Page  VII-24,  Table  VII-6:  We  agree  with  the  comment  on  tactical  missiles  and 
have  modified  the  report  to  reflect  this  information.  We  note  the  comments  on 
simulators/trainers  and  the  various  models,  but  we  stand  by  our  estimate. 

9.  Page  VII-23,  Cld:  Noted.  We  stand  by  our  estimate.  We  assume  that  there  will  be 
weapon-system-speciflc  information  in  the  manuals.  Also,  the  manuals  will  reflect 
enhancements  in  functionality. 

10.  Page  VII-26,  Section  Clfl:  We  agree  and  have  modified  the  report  to  reflect  this 
information. 

1 1 .  Page  VII-26,  Section  C 1  f2:  Noted. 

12.  Page  Vn-27,  Section  Clf3:  Noted.  We  stand  by  our  estimate. 
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13.  Page  Vn-27,  Section  Clf4:  Noted.  We  stand  by  our  estimate.  We  expect  that  there 
will  be  additional  DIREPs  during  deployment.  This  category  reflects  those 
DIREPs. 

14.  Page  VII-27,  Section  ClfS:  Noted.  We  stand  by  our  estimate. 

15.  Page  VII-28,  Section  C2al:  Noted.  We  applied  the  same  percentage  across  all 
three  systems. 

16.  Page  Vn-28,  Section  C2c:  We  applied  the  same  labor  hours  per  year  across  all 
three  systems. 

1 7 .  Page  VII-29,  Section  C2d,  paragraph  1 :  We  agree  and  have  modifled  the  report  to 
reflect  this  information. 

18.  Page  VII- 30,  Section  C2el,  paragraph  2:  We  applied  the  same  criteria  across  all 
three  systems. 

19.  Page  VII-30,  Section  C2e2:  We  agree  and  have  modified  the  report  to  reflect  this 
information. 

20.  Page  VII-30,  Section  C2e3:  We  stand  by  our  estimate,  which  used  the  same 
criteria  across  all  three  systems. 

2 1 .  Page  VII-30,  Section  C2e4;  Noted. 

22.  Page  Vn-31,  Section  C4:  We  agree  and  have  modified  the  report  to  reflect  this 
information. 

23.  Page  VII-31,  Section  Dl,  paragraph  3:  We  agree  and  have  modified  the  report  to 
reflect  this  information.  In  the  final  report,  we  have  treated  all  three  systems  the 
same. 

24.  Page  Vn-33,  Table  VH- 10:  Noted. 

25.  Page  VII-36,  Table  VII-1 1:  We  agree  and  have  modifled  the  report  to  reflect  this 
information. 
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RESPONSE  TO  IDA  "DRAFT"  REPORT  -  COSTS 


1.  Page  VII-1,  paragraph  6 

The  report  uses  1920  hours  in  its  cost  analysis.  DRC 
charges  the  government  1820  hours  per  year.  Our  Andover 
average  burdened  labor  rate  is  $81,445  per  year  and  the 
Field  Rep  burdened  labor  rate  is  $54,000  per  year.  This 
becomes  significant  for  the  period  of  the  study. 

2.  Page  VII-2,  Section  A,  Cost  Estimate  for  CAMS 

DRC  was  upfront  with  its  cost  data  and  we  are  concerned 
that  some  data  for  CANS  costs  were  not  made  available  in 
time  for  the  publication  of  the  draft  IDA  report.  This 
creates  a  degree  of  uncertainty  because  now  there  is  an 
awareness  of  what  the  target  cost  have  to  be. 

3.  Page  VII-5,  Section  Alb,  paragraph  4 

The  feasibility  of  providing  an  enhanced  editing 
capability  with  immediate  communications  between  CAMS  and 
RENIS  has  not  been  addressed.  Will  current  technology 
support  that  type  of  interface? 

4.  Page  vll-8.  Section  A2e,  paragraph  4 

DRC  questions  the  requirement  for  only  four  experienced 
database  managers,  all  located  at  SSC.  Given  the 
inherent  problems  associated  with  providing  support  for 
this  type  of  a  remote  database,  it  seems  logical  that 
more  personnel  than  that  required  for  TICARRS  would  be 
appropriate.  DRC's  own  estimate  for  database  managers  is 
7-8.  The  cost  estimates  for  experienced  DBMS  at  SSC 
seems  excessively  low.  DRC  does  not  see  any  costs  for 
assistance  provided  by  UNISYS  personnel  at  SSC.  where 
are  the  costs  for  database  managers  at  the  base  level 
included? 

5.  Page  vll-16.  Section  C,  paragraph  1 

DRC  believes  that  the  scheduled  implementation  of 
TICARRS-92  for  the  entire  Air  Force  can  be  accelerated  by 
several  months.  Cost  differentials  for  the  DRC  proposed 
implementation  schedule  are  included  as  part  of  the  DRC 
costing.  DRC  would  assume  a  contract  start  date  of 
January  1,  1994.  We  would  agree  with  the  IDA  comment 
that  six  months  would  be  spent  in  enhancing  the 
functionality  of  TICARRS-92  to  take  over  the  functions  of 
CAMS/REHIS.  The  enhancement  period  would  be  January 
1-June  30,  1994.  DRC  also  agrees  that  the  TICARRS 
database  would  be  initialized  for  one  weapon  system,  the 
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F-117A,  and  CAMS  data  from  one  base,  Holloman  AFB,  NM, 
would  be  loaded  onto  TlCARRS-92.  This  would  be 
accomplished  by  July  1,  1994.  The  OT&E  would  then  occur 
during  July  through  September  1994  at  Holloman  AFB  for 
the  F-117A.  During  the  period  of  time  from  January  1 
through  September  1994,  DRC  would  begin  the  "Activities 
required  for  Weapon  System  Initialization”  and  the 
"Activities  required  for  Unit  Activation".  Weapon  system 
initialization  would  include  activities  such  as 
determining  weapon  system  peculiar  data,  i.e.  WUC, 
Configuration  items.  Job  standards,  etc.  Unit  activation 
initialization  would  include  completing  a  site  survey  and 
obtaining  information  such  as  serial  numbers,  TCTO  header 
information,  etc.  Sufficient  weapon  systems  and  bases 
would  be  completed  by  October  1,  1994  to  support  training 
those  locations  and  weapon  system  personnel.  As  training 
is  proceeding,  the  remaining  weapon  systems  and  bases 
would  be  initialized  with  all  initialization  and  training 
cpmpleted  not  later  than  September  30,  1995.  This 
schedule  would  allow  the  shut-down  of  CAMS/REHIS  several 
months  earlier  than  the  IDA  report  anticipates  at  a 
substantial  cost  savings  to  the  Government. 

6.  Page  VII-19,  Section  Clb,  paragraph  1 

Functional  Enhancements.  DRC  has  a  well  established 
track  record  of  completing  all  efforts  within  budget  when 
they  are  estimated  at  the  level  of  detail  provided  for 
these  enhancements.  DRC  will  have  PQDR  completed  before 
Oct  1993  and  the  CEHS  interface  requirement  should  only 
include  software  to  support  an  interface  with  CEHS  and 
not  to  replicate  all  CEHS  processing.  This  is  estimated 
as  Option  2.  Therefore,  we  have  reduced  the  estimate  to 
a  total  of  15,134  manhours  or  $.8m  using  DRC's  Andover 
labor  rate. 

7.  Page  VIl-21,  Section  Clb,  paragraph  8 

System  Software  Interfaces.  Of  the  thirteen  systems 
listed  as  having  interfaces  with  REHIS,  eight  receive 
data  from  REHIS,  one  provides  data  to  and  receives  data 
from  REHIS  and  one,  D087F,  apparently  has  no  interface 
with  REHIS.  GOOIC  will  be  replaced  by  TICARRS-92, 
because  depot  users  will  input  their  data  on-line  into 
TICARRS-92.  Of  the  remaining  system  interfaces, 
TICARRS-92  will  be  required  tO  generate  nine  output  files 
and  process  data  from  seven  input  files.  Both  the 
generated  files  and  the  input  files  are  straight  forward 
extracts  of  data  from  the  data  base  or  the  loading  of 
reference  data  or  other  types  of  data  (status, 
utilization,  and  HDC  data  from  G081)  which  TICARRS-92  can 
already  load.  DRC's  estimate  of  $560,000  for  developing 
the  interfaces  provides  for  approximately  800  manhours 
per  file.  As  our  view  of  the  actual  requirements  becomes 
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clearer,  it  appears  to  be  an  estimate  that  we  would  have 
great  difficulty  exceeding. 

8.  Page  VII-24,  Table  VII-6 

DRC's  original  estimates  by  weapon  system  included  G&A 
and  Profit.  The  summary  costs  included  in  the  10  year 
cost  projections  excluded  the  G&A  and  Profit  values. 

DRC  has  added  costs  to  activate  the  B2  on  TXCARRS. 

DRC  has  completed  additional  research  on  the  issue  of 
activating  Tactical  Missiles  on  TICARRS.  Everyone  we 
have  contacted  has  come  back  with  basically  the  same 
information.  Information  on  Tactical  Missiles  is 
reported  using  the  Tactical  Missile  Reporting  System 
(TMRS).  Personnel  responsible  for  the  Tactical  Missiles 
do  not  have  CAMS  terminals.  When  they  need  to  order 
parts  for  the  missile  trailers,  they  may  go  to  a  shop 
that  has  a  CAMS  terminal  and  use  the  CAMS/SBSS  interface. 
l£  they  need  some  support  from  a  backshop,  they  may  use  a 
CAMS  terminal  to  generate  a  WCE  using  a  generic  WUC. 
Personnel  and  Training  Management  data  is  loaded  into 
CAMS  for  these  personnel.  Providing  for  the  above 
mentioned  requirements  can  be  accomodated  in  TICARRS 
without  any  additional  cost.  The  first  two  requirements 
would  impact  operational  cost  if  they  happened 
frequently,  but  we  have  been  told  that  is  not  the  case. 
Since  we  plan  an  automated  load  of  all  MPT  data,  the 
volume  of  records  loaded  for  a  unit  will  not  impact  the 
costs.  Therefore  there  are  no  additional  costs  included 
for  Tactical  Missiles. 

Simulators/Trainers,  were  already  included  in  DRC's  cost 
projections  as  part  of  the  weapon  system  and  unit 
activation  costs. 

DRC's  previously  provided  weapon  system  estimates  already 
take  into  account  the  various  models  and  mission 
configuration  requirements.  For  example,  with  the  C130 
the  mission/debrief  is  rated  with  a  complexity  of  5 
because  we  will  need  to  make  modifications  to  debrief. 

DRC  does  not  agree  with  the  20%  delta  for  the 
Block/Models  for  the  F-16  and  F-15.  All  the  necessary 
information  already  resides  in  TICARRS.  The  cost 
estimates  for  initializing  each  weapon  system  provided  by 
DRC  took  into  account  the  various  models  and  mission 
configuration  impacts.  Also,  the  cost  estimates  provided 
by  DRC  included  fully  burdened  labor  plus  G&A  and  Profit. 
For  the  F-16  weapon  system,  the  Block  40  and  50  aircraft 
are  the  equivalent  of  adding  additional  models  of 
aircraft.  These  blocks  have  their  own  Work  Unit  Code 
(WUC)  and  test  station  manuals.  Block  10  aircraft  are  A 
and  B  models.  Block  10,20,  and  30  aircraft  all  use  the 
same  test  stations.  All  of  the  basic  information 
(Organization  structure.  Equipment  Ids,  WUCs,  Part 
Numbers,  Configuration  Templates,  Phase  Dock  Inspection 
Decks,  etc)  needed  to  activate  the  entire  F-16  weapon 
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system  have  already  been  loaded  to  the  TICARRS  data  base. 
The  bulk  of  the  hours  estimated  for  the  activation  is  to 
verify  that  the  information  is  correct  as  loaded  and  to 
load  any  additional  information,  such  as  TCTOs,  JSTs,  and 
AMOC  information  to  the  system.  A  particular  TCTO  may 
relate  to  one  or  more  models,  but  the  fact  that  it  does, 
will  not  impact  the  weapon  system  initialization  cost. 

The  volume  of  data  is  what  determines  the  initialization 
costs. 

For  the  F-15  weapon  system,  costs  have  been  included  for 
building  configuration  templates  for  models  A,  B,  C,  and 
D.  They  have  already  been  built  for  the  E  model  as  part 
of  the  Operational  Assessment  at  SJAFB.  The  data  load 
for  the  F-15  weapon  system  is  actually  estimated  higher 
than  for  the  F-16  because  there  is  less  data  loaded  for 
the  F-15,  even  though  there  are  more  F-16  models. 

For  the  C-130  weapon  system,  tankers,  helicopters,  etc., 
DRC  can  identify  no  reason  to  add  additional  costs  for 
each  type  of  aircraft.  DRC  stands  by  its  original 
estimates  for  all  these  weapon  systems. 

9.  Page  VII-23,  Cld 

The  costs  for  producing  and  distributing  the  initial 
release  of  the  user  manual  is  not  related  to  the  cost  of 
adding  additional  functionality,  establishing  interfaces 
with  other  systems,  and  initializing  weapon  systems.  It 
is  strictly  a  factor  of  the  number  of  copies  that  have  to 
be  produced  and  distributed.  DRC  stands  by  its  original 
estimate  of  $600k. 

10.  Page  VII-26,  Section  Clfl 

DRC  agrees  with  the  manhour  estimate  but  the  Andover 
Labor  rate  should  have  been  used,  not  the  Field  Rep  rate. 

11.  Page  VII-26,  Section  Clf2 

DRC  stands  by  its  original  estimate. 

12.  Page  VII-27,  Section  Clf3 

The  cost  of  the  site  representatives  that  will  be  used 
for  the  user  training  are  costed  by  DRC  at  $54,000  per 
year^  Therefore,  the  totals  for  this  section  should  be 
$1.84M. 

13.  Page  VII-27,  Section  Clf4 

DIREPS  are  considered  by  DRC  to  be  a  part  of  the  O&H 
costs.  The  charge  for  software  maintenance  during  the 
same  period  of  time  includes  anticipated  cost  for  DIREP 
resolution. 
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14.  Page  VII-27,  Section  Clf5 

This  cost  should  not  be  included  as  an  hourly  cost.  The 
short-term  support  would  be  provided  by  a  site 
representative  already  on  the  DRC  payroll.  The  only 
possible  cost  in  this  category  would  be  per  diem  costs  at 
a  rate  of  $100.00  per  day.  That  rate  should  be  by  base 
and  not  by  unit.  The  cost  of  this  short-term  support 
should  be  $228,900  rather  than  $1.1M. 

$100  per  day  x  21  days  x  109  bases  «*  $228,900. 

15.  Page  VII-28,  Section  C2al 

The  contractor  provides  management  for  his  program.  The 
SPO  staff  can  be  much  smaller  than  if  it  was  managing  an 
organic  program. 

16.  Page  VII-28,  Section  C2c 

The  1920  hours  per  year  are  too  high.  DRC  costs  for  site 
representatives  are  calculated  at  an  average  of  $54,000 
per  year.  Based  upon  those  costs,  the  User  Support  costs 
should  be  as  follows: 

FY  1994  -  34  site  representatives 

FY  1995  -  Approximately  67  site  representatives 

FY  1996  thru  FY  2003  -  112  site  representatives 

17.  Page  vlI-29,  Section  C2d,  paragraph  1 

The  communications  costs  should  be  recalculated: 

Costs:  (based  upon  SPRINT  rates) 

-  Upgrade  from  9.6  Kbps  to  19.2  KBPS  is  $750  per 
line,  a  one-time  charge  per  line. 

$750  X  57  -  $42,750 

-  The  cost  of  a  19.2  Kbps  line  is  $1145  per  month. 

-  The  cost  of  a  56  Kbps  line  is  $1750  per  month. 

The  annual  communications  costs  should  be: 

-  107  sites  X  $1175/month  x  12  months  -  $1,470,180 

-  2  56  Kbps  lines  at  $1750/month  x  12  months  - 

$105,000 

-  Total  projected  annual  Communications  Costs  * 

$1,554,930 


Lines  to  RPC's: 

These  costs  should  be  eliminated  because  they  are  costed 
under  the  Comm  charges  (2.4.1). 

18.  Page  VII-30,  Section  C2el,  paragraph  2 

The  estimation  criteria  of  25,000  LOC  per  maintainer  has 
no  basis  from  the  DRC  perspective.  When  an  organization 
is  given  responsibility  for  both  operating  and 
maintaining  a  software  application  the  staffing 
requirements  cannot  be  dependant  solely  on  the 
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application  lines  of  code.  The  staff  requirements  for  an 
online  system  are  not  the  same  as  for  a  batch  fed  system. 
An  on-line  system  requires  personnel  dedicated  to  quickly 
responding  to  DIREPs  and  other  personnel  dedicated  to 
monitoring  the  performance  of  the  data  base  and  the 
transaction  processor.  A  batch  fed  system  does  not  have 
the  same  response  time  requirements  and  therefore  a 
smaller  staff  can  be  assigned  to  these  functions.  A 
batch  fed  system  requires,  however,  a  staff  to  ensure 
that  all  data  is  received  and  processed,  a  staff  which  is 
not  needed  with  an  on-line  system.  Therefore  estimating 
staff  requirements  must  consider  the  staffing  experience 
when  TICARRS  was  an  on-line  system.  Also,  TI CARRS  LOC 
will  not  increase  significantly  as  each  weapon  system  is 
added  so  DRC  staff  estimates  cannot  be  based  solely  on 
LOC. 

This  section  includes  all  costs  associated  with  ensuring 
that  the  TICARRS  application  operates  well.  It  includes 
data  base  administrators  and  TP  administrators,  as  well 
as  personnel  for  OIREP  support,  software  improvements, 
and  production  support. 

19.  Page  VII-30,  Section  C2e2 

Database  Management  for  TICARRS  includes  a  variety  of 
user  support  activities,  some  of  which  do  not  seem  to  be 
accounted  for  in  the  REMIS  and  CAMS  estimates.  Software 
QA,  Help  functions  and  documentation  maintenance  are 
included,  but  the  costs  for  updating  reference  type  data 
do  not  seem  to  be  included  for  CAMS  and  REMIS. 

20.  Page  VII-30,  Section  C2e3 

Based  on  our  past  experience  software  maintenance  rarely 
results  in  a  need  to  update  documentation.  Therefore,  we 
have  revised  this  to  two  percent. 

21.  Page  vll-30,  Section  C2e4 

Travel  is  not  required  for  training.  It  is  primarily 
used  for  Program  Management  Reviews  and  coordination 
meetings  with  both  Air  Force  and  Field  Rep  personnel. 

22.  Page  VII-31,  Section  C4 

G&A  Plus  Profit.  This  should  be  calculated  on  all 
contractor  costs,  but  not  on  government  costs.  ORC's 
estimate  for  this  category  has  included  only  contractor 
items. 

23.  Page  VII-31,  Section  Dl,  paragraph  3 

Why  are  facilities  costs  only  included  for  the  TICARRS 
PMO?  They  should  probably  be  included  for  all  military 
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and  civilian  personnel. 

24.  Page  VII-33,  Table  VII-10 

Task  2.5.1  is  caluculated  incorrectly.  Tasks  3.5.5 
should  be  reduced  consistent  with  2.5.1.  Tasks  2.3.1  and 
3.5.4  should  also  be  reduced  as  the  number  of  bases  and 
number  of  software  maintainers  are  reduced.  DRC's 
revisions  to  this  table  are  attached. 

25.  Page  VII-36,  Table  VII-11 

REHIS  software  maintenance  requirements  should  decrease 
similar  to  that  estimated  for  CAMS.  DRC's  revisions  to 
this  table  are  attached. 

DRC  is  also  attaching  revisions  for  Tables  Vll-4  and 
VlI-9. 
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DRC  TICARRS  Table  VII-4 


1  IDA 

1994  1995 

1996 , 1997 ! 1998 

!  1999  1  2000  i  2001  2002  2003  Total  Total  Delta 

1.0  ‘Non-recurring  Costs 


1.1  !  Hardware 


1.1.1  1  Mainframes 

!  6.60 

1  6.60 

1 

1.2  [Software  ! 

i 

I  1.75  1.761 


16.70  0.00 


1.2.1  lApplication  Software 


IFunct'l  Enhancements 


I  System  Interfaces 


1 .2.2 1  Database  Initialization 


1 .2.3  I  Documentation  (10%) 


1 .3  I  Communications 


1 .3.1  I  Hardware 


1.4  ! T raining/Unit  Activ 


1.4. 2 1  Load  CAMS  Data 


1 .4.3 1  User  T raining 


0.80 


I  0.56 


I  2.35  3.44 


i  0.301  0.30  0.00 


I  ! 

0.051 

0.05 

!  0.00 

1. 4.4:  Resolve  DIREPs 

I  0.00  j 

0.00 

1 .4.5  Sho'rt-Term  Support 

I  O.OOI 

0.00 

1 .4.6 'Travel  (Training)  1 

I  2.10I 

0.00 

1. 4. 7 'T&L  (Short-Term) 

1.5  I  Travel  (Unit  Activation)  | 

■  Subtotal  Non-recurrin 


2.0  I  Recurring  Costs 


2.1  I  Program  Management 


2.1.1 1  Government  * 


2. 1.2 'Contractor 


2.2  ‘  Computer  Operations 


2.2.1  ■  Central  Computer 


■  Operational  Staff 


I H/W,  S/W,  Utilities 


2.2.2 'Base  Comm  Operation  * 


I  CIV  &  Mil  Personnel 


H/W,  S/W,  Utilities 


I  1.361  2.191  1.561  1.561  1.561  1.561  1.66|  1.561  1.56 


I  0.65  1.04  0.75  0.751  0.751  0.751  0.751  0.751  0.751  0.75 


0.70i  1.40  2.221  2.22  2.22 


1.891  1.891  1.891  1.891  1.891  1.89 


2.2.3 1 OCONUS  Operations  *  { 

!  1 

1 

■  CIV  &  Mil  Personnel  :  0.741 

2.86| 

4.46 

i  H/W,  S/W  Utilities 


■  User  Support 


Communications 


.1  Comm  Charges 


.2  Lines  to  RPCs 


Software  Maintenance 


.1  Application  Software 


.2  Database  Management 


.3 1  Documentation  (2%) 


i  Training  &  Travel 


I  Subtotal  for  Recurring 


IG&A  Plus  Profit 


■  T  otal 


■  Non-contractor  (*  items) 


1.111  1.11  15.781 


7.65I11.38I11.38111.38I11.38  11.38  11.38 


2.401  2.40!  2.401  2.40|  2.40|  2.401  2.401  2.401  2.40!  22.40: 


lUBcMlUcIjailTill 


4.461  4.461  4.461  4.46;  4.46  {  39.281 


8.8 


91.13, 


21.601 


36.331 


i  0.251  0.751  0.981  0.98 i  0.981  0.981  0.981  0.981  0.98!  0.98 


'  1.84  3.58:  6.05  6.051  6.051  6.051  6.05 


0.85!  l.58i  1.58'  1.581  1.581  1.581  1.58]  1.58|  1.581  1.58 


0.00  0.00  0.00'  o.oo:  O.OOl  O.OOl  O.OOI  o.oo!  O.OOi  0.00 


3.20  4.061  4.271  4.27!  4.27  4.27 


1.301  1.93:  I.93I  1.93!  1.93!  1.93 


0.06;  0.08i  0.09  0.09  0.09 


;  0.20!  0.30'  0.35I  0.35I  0.35 


13.84  31.71  |39.90139.90!39.90 


3.451  4.841  2.87 


4.271  4.271  4.27!  4.27 1 


1.93;  1.93!  1.93^  1.93; 


0.09  0.09 1  0.09'  0.09! 


0.35  0.35 


3.01  3.01  2.75 


41.41 


18.67 


0.83 


3.30 


29.59 1 52.92 1 42.77 1 42.77 1 42.77)  42.77 


BE0 

nciurgiTFii 


41.87  41.87  425.11 


15.88! 


1.99' 


41.45 


11.161 


2.63: 


5.521 


367.95 


59.15 


474.76 


3. 1 5 !  1 5.85 !  20.78 !  20.78 1 20.78 1 20.78 1 20.78 1 20.78 1 20.78 1 20.78 1 1 85.28 1 


-0.04 


7.51 


-1.80 


-2.22 


-8.29 


-27.87 


-49.65 
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TICARRS  -  Two  Year 


IDA  -  Alternative  2  •  Table  VII -9 

;  1994  1995 

1996  1997  1998 

1999  i  2000  '  2001  ,  2002  2003  Total 

CAMS 

48.061  34.65 

20.49,  0.00!  0  00 

0.00!  0.00;  O.OOi  0.00'  0.00  103.20 

REMIS 

10.66 

10.66 

10.66'  0.00 

0.00 

O.OOI  0.00 

0.00 

0.00'  0.00  31.98 

TICARRS 

29.51 

51.40 

52.541  48.73 

48.73 

48.73 

49.85 

49.59 

47.84'  47.84  474.76 

! 

i 

Total 

88.23 

96.71 

83.69 

48.73 

48.73 

48.73 

47.84!  47.84  609.94 

Present  Value 

88.23 

92.72 

76.93 

42.95 

41.18 

39.48 

38.72 

36.93 

34.16'  32.75 i  524.06 

1 

1 

i 

_ _ 

_ 

DRC  -  Alternative  2  •  Table  VII-9 

(  1 - - - 

1  1 

1  1  1994 

1999 

2000 

2001 

2002  !  2003  1  Total 

CAMS 

45.74 

18.81 

HEEI 

0.00 

0.00 

0.00 

0.00 

0.00 

REMIS 

0.00 

0.00 

0.00 

0.00 

O.OOi  O.OOI  16.20 

TICARRS  1  29.59 

52.92 

42.77' 

42.77 

42.77 

42.77 

43.89 

43.89 

41.87i  41.871  425.11 

1 

i 

I  1 

1 

^ ^ - 

Total  1  85.05 

78.22 

42.771  42.77 

42.77 

42.77 

43.89 

43.89 

41.871  41.871  505.86 

Present  Value  1  85.05 

74.99' 

34.65 

34.09 

32.691 

29.901  28.66!  433.18 
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Alt  2  CAMS  Table  VII-10 


)  I  Non-recurring  Costs 


I  I  Hardware 


1.1  [CAMS  share  of  RPC  H/W 


!  !  Software 


[.1  Application  Software 


{Approved  CRSD 


I  Enhanced  Editing 


New  Weapons/Equip 


1.2.3  Documentation 


1 .3  Communications 


1 .4  {Training 


on-recurring  Sub-Total 


2.0  'Recurring  Costs 


2.1  ! Program  Management 


2.1.1  I  Government 


2.2  i Computer  Operations 


2.2.1  I  RPC  Operations 


Civ  and  Mil  Personnel 


H/W,  S/W,  Utilities 


2.2.2IOCONUS  Operations 


!  Civ  and  Mil  Personnel 


I  H/W,  S/W,  Utilities 


2.2.3 1  Base  Comm  Operations 


I  Civ  and  Mil  Personnel 


I  H/W,  S.W,  Utilities 


2.3  I  User  Support 


2.3.1  I  Field  Help  Group 


2.3.2 iBase  Representatives 


2.4  I  Communications 


2.4.1  iRPC  H.S.  Base  Links 


2.4.2  High  Speed  REMIS  Link 


2.5  Software  Maintenance 


2.5.1  Application  Software 


3.5.4  Software  Maint  Operations 


3.5.5  'System  Int  and  Test 


2.5.2  {Database  Management 


2.5.3  i  Documentation 


2.6  Training  &  Travel 


{Recurring  Costs  Sub-Total 


Total  Costs 


1994  11995!  1996  TOTAL 


!  1994  1995  1996  TOTAL  DELTA 


System  Int  and  Test 

1.2.2 

Initialize  Database 

0.361  0.27  0.19 


0.16 

0.09 

0.00 

0.251 

-0.57 

0.40 

2.43 

1.08 

6.82 

1.221  0.401  0.00 


3.59 


1.62 

-0.81 

4.671 

-2.15 

10.021  7.521  5.01  22.55 


2.661  1.99  1.331  5.98 


16.88111.251  5.621  33.75 


8.001  6.00  3.001  17. 


10.02 

5.01 

1  0.00] 

15.03 

1  -7.52 

2.66 

1.33! 

!  0.00  i 

3^^ 

-1.99 

0.171  0.17  0.171  0.51 


0.171 

0.091 

O.OOi  0.26i 

1  -0.26 

0.56810.749  0.771  2.09 


I  3.11,  2.331  1.55i  6.99' 


0.52;  0.52:  0.52!  1.56 


;  0.65i  0.651  0.65  1.95 


!  0.171  0.1 3.i  0.091  0.39 


0.09  0.24 


0.07  0.05  0.03  0.15 


20.50  103.23 


148.08:34.65  20.50!  103.23 


I.IOi  0.651  0.00 


0.52 i  0.26!  0.00 


0.55:  0.331  0.00 


0.17 


0.08 


0.06 


45.74 


45.74  18.811  0.00 


1.751 

1  -5.25 

0.78 

i  -0.78 

0.88 

1  -1.08 

0.271 

-0.12 

0.12 

-0.12 

0.11' 

-0.04 

64.56 

1  -38.67 

64.56 

!  -38.67 

I 

I 

V-17 

I 


7/23/93  3:40  PM 


IDA-R2.XLS 


Alt  2  REMIS  Table  VII-1 1 


1994  1995  1996  TOTAL 

'1994  1995  1996  total  DELTA 

1.0  i  Non-recurring  Costs  1  ; 

1.1  1  Hardware  ' 

1  '  ■ 

1.1.1  !  Mainframes  I  1  ' 

_ ^ ^ ^ _ _ 

2.0 


2.1 


2.1.1 


2.1.2 


I  Software 


I  Application  Software 


Improve  Performance 


I  Improve  REMISTALK 


I  Initial  Database 


Documentation 


Communications 


Trainin 


Subtotal  Non-recurring 


)  Recurring _ 


Program  Management 


'Government 


I  Contractor 


2.2  I  Computer  Operations 


2.2.1 1  Central  Computer 


Operational  Staff 


!H/W,  S/W,  Utilities 


2.2.2 1  At  ALCs 


[Operational  Staff 


iH/W,  S/W,  Utilities 

I  0.311 

I  0.31 

0.31 

2.3  User  Support 

i  0.92 

;  0.92 

0.92 

2.4  Communications 

!  0.16 

■  0.16 

0.16 

2.5  [Software  Maintenance 

! _ ^ ’ 

2.5.1  [Application  Software  1 

1  3.461 

!  3.461 

3.46 

2.5.2  Database  Management 


2.5.31  Documentation 


2.60  [Training  &  Travel 


Subtotal  for  Recurring 


2.0  IG&A  Plus  Profit 


Total 


0.20  0.20  0.20 


0.42 


1  9.271  9.27  9.27 


1.391  1.39  1.39 


i  10.66'  10.661 10.661 


10.38 


1.74 


0.60 


1.26 


27.81 


4.17 


31.981 


0.31  0.23  0.00 


0.78 


0.16  0.121  0.00 


2.941  1.731  0.00 


0.58 


0.171  0.101  0.00 


8.54 


1.18 


9.72i  6.491  0.00 


1.241 

i  -1.52 

0.28 

•0.20 

4.671  -5.71 


0.27! 

i  -0.33 

0.67 

1  -0.59 

14.24 

i  -13.57 

16.20'  -15.78 


V-I8 


C.  GENERAL  COMMENTS 

DRC  Comments: 

A  copy  of  the  general  comments  from  DRC  (referred  to  as  “Tab  B”  in  the 
forwarding  letter)  appears  beginning  on  page  V-23  of  this  document.  The  page  numbers  in 
both  the  DRC  comments  and  the  IDA  responses  refer  to  page  numbers  in  the  review 
version  of  EDA  Paper  P-2863. 

IDA  Response: 

1 .  Page  I-l,  Section  a,  paragraph  7:  Very  few  officers  participated  in  the  surveys; 
however,  we  agree  and  have  modified  the  report  to  reflect  this  information. 

2.  Page  II-8,  Section  2,  paragraph  1:  We  agree,  although  the  CAMS/REMIS  PMO 
indicates  that  development  of  CAMS  user  access  to  REMIS  through  CAMS 
terminal  is  under  way.  No  credit  has  been  given  for  this  activity  at  this  time. 

3.  Page  II-9,  Section  Cl,  paragraph  1;  We  agree  and  the  report  has  been  modified  to 
reflect  this  information. 

4 .  Page  n- 1 1 ,  section  2,  paragraph  2:  Chapter  II  has  been  revised. 

5.  Page  IM 1,  section  2,  paragraph  3:  Noted,  but  this  is  more  extensive  information 
than  we  wanted  to  put  in  Chapter  II. 

6.  Page  11-12,  Section  3c,  paragraph  1:  Systems  in  development  are  not  prominently 
noted  in  the  IDA  report 

7.  Page  III-2,  Section  1,  paragraphs  2, 3, 4:  Noted.  We  have  treated  the  functions  of 
the  systems  more  extensively  in  Chapter  V. 

8.  Page  III-3,  Section  2,  paragraph  2;  Noted. 

9.  Page  III-5,  Section  a,  paragraph  1;  Noted.  The  relevant  section  discusses 
requirements  only. 

10.  Page  III-9,  Section  5b,  paragraph  4:  Noted.  We  have  addressed  the  extent  of  effort 
necessary  later  in  the  report 

11.  Page  III- 10,  Section  d,  paragraph  3;  We  agree  and  have  modified  the  report  to 
reflect  this  information. 

12.  Page  ni-12.  Section  B,  paragraph  3:  We  have  considered  past  records  on  meeting 
cost  and  schedule  objectives  in  making  our  estimates. 

13.  Page  IV-6,  Section  E:  We  have  included  a  discussion  of  the  Operational 
Assessment  and  IDA’s  role  in  it  but  we  do  not  believe  that  the  MOA  needs  to  be 
appended.  It  is  part  of  the  public  record. 
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14.  Page  IV-9,  Section  4,  paragraph  2:  We  agree  and  have  modified  the  report  to 
refiect  this  information. 

[Note:  Number  15  was  omitted  from  DRC’s  numbering  scheme.] 

16.  Page  V-3/4,  Table  V-1:  Some  of  these  points  have  been  incorporated  into  the  table 
and  others  have  been  dealt  with  in  the  text  of  the  final  report 

17.  Page  V-3,  Table  V-1:  The  table  notes  only  the  active  planning  and  scheduling 
activities  such  as  is  done  in  CAMS.  The  text  reflects  differences  in  approaches. 

18.  Page  V-3,  Table  V-1 :  The  table  has  been  modified  to  reflect  this  information. 

19.  Page  V-4,  Table  V-1:  The  issue  is  that  TICARRS  only  records  the  information;  it 
must  be  entered  and  changed  by  the  technician.  CAMS,  through  an  interface  with 
SBSS,  can  query  the  system  and  have  the  information  automatically  retrieved. 

20.  Page  V-4,  Table  V- 1 :  TICARRS  requires  a  special  computer  run  to  trace  multiple 
cannibalizations  or  one  can  browse  through  maintenance  history.  CAMS  provides 
a  query  in  background  to  provide  the  information.  The  Operational  Assessment 
revealed  this  inability  to  track  multiple  cannibalizations  within  TICARRS  to  be  a 
deficiency.  The  text  has  been  modified  to  reflect  this  information. 

2 1 .  Page  V-4,  Table  V-1:  We  agree  that  it  supports  but  does  not  have  an  “active”  shop 
scheduling  function,  which  is  among  DRC  enhancements  for  TICARRS. 

22.  Page  V-4,  Table  V-1:  The  report  has  been  modified  to  reflect  this  information. 

23.  Page  V-5,  Section  3a,  paragraph  1:  At  this  time,  the  deficiency  must  be  noted  and 
its  costs  estimated. 

24.  Page  V-6,  Section  3c,  paragraph  1:  The  report  has  been  modified  to  reflect  this 
information. 

25.  Page  V-6,  Section  3c,  paragraph  2:  This  was  difficult  at  the  Operational 
Assessment,  but  we  agree  and  the  table  has  been  modified  to  include  this 
information. 

26.  Page  V-8,  Section  3g,  paragraph  3:  The  point  here  is  that  the  technician  cannot 
close  the  job.  TICARRS  has  not  demonstrated  the  ability  to  create  jobs  involving 
multiple  work  centers. 

27.  Page  V-8,  Section  3g,  paragraph  6:  The  TICARRS  ARC  does  not  support  the  Air 
Force  in  the  same  manner  as  the  ARC  in  CAMS.  This  may  be  a  matter  of  policy 
and  procedures  and  have  little  substance  beyond  reporting,  but  it  was  serious 
enough  at  Seymour  Johnson  AFB  to  warrant  using  CAMS  in  place  of  TICARRS 
for  this  function.  If  the  problem  were  limited  to  data  in  one  place  as  indicated  in  the 
DRC  response,  it  seems  as  though  DRC  and  the  4th  Wing  would  have  fixed  that 
problem  and  proceeded  to  use  TICARRS.  They  did  not 
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28.  Page  V-29,  Section  3g,  paragraph  7:  TICARRS  does  not  provide  narratives  for 
look  Phase,  only  for  fix  Phase  and  TCIs. 

29.  Page  V-10,  Section  3g,  paragraph  12:  TICARRS  PQDR  capability  is  limited  to  the 
F-16  community  and  F-16  formats  for  interface  with  INFOCEN.  The  F-15  and 
F-16  data  bases  are  not  standardized  in  INFOCEN,  and  changes  to  TICARRS 
would  be  required  to  interface  on  other  weapon  systems.  The  issue  of  whether 
originators  can  make  changes  after  submission  to  QA  will  arise  if  TICARRS  is  to 
support  the  rest  of  the  Air  Force.  Thus,  the  response  by  DRC  is  correct  but  does 
not  negate  the  necessity  of  making  changes  to  TICARRS  to  process  PQDRs 
differently  from  current  capabilities. 

30.  Page  V- 10,  section  3h,  paragraph  2:  We  disagree. 

31.  Page  V-1 1,  Section  i,  paragraph  2:  TICARRS  92  failed  to  collect  and  track  all 
engine  maintenance  information  within  the  Engine  Maintenance  shop  at  Seymour- 
Johnson  even  after  engine  configuration  problems  were  solved.  These  problems 
arose  at  the  very  end  of  the  assessment.  In  addition,  TICARRS  treats  on- 
equipment  maintenance  as  aircraft  maintenance  attaching  LRUs  to  the  aircraft 
instead  of  to  the  engine.  Engine  management  requires  that  the  engine  be  treated  as 
an  end  item  for  all  maintenance.  When  an  engine  is  removed,  the  LRUs  must  move 
with  the  engine. 

32.  Page  V-1 1,  Section  j,  paragraph  1:  The  report  has  been  modified  to  reflect  this 
information. 

33.  Page  V-1 1,  Section  j,  paragraph  2:  TICARRS  requires  a  special  computer  run  to 
trace  multiple  cannibalizations  or  one  can  browse  through  maintenance  history. 
CAMS  provides  a  query  in  background  to  provide  the  information.  The 
Operational  Assessment  revealed  this  inability  to  track  multiple  cannibalizations 
within  TICARRS  to  be  a  deficiency.  The  engine  issue  was  supposed  to  be 
uninstalled  to  installed  engine.  The  text  has  been  modified. 

34.  Page  V-13,  Section  ck,  paragraph  2:  We  have  been  unable  to  verify  this  purported 
problem.  Kit  status  in  TICARRS  is  monitored  only  if  the  information  is  manually 
entered  and  manually  tracked. 

35.  Page  V-14,  Section  p,  paragraph  2:  The  report  has  been  modified  to  reflect  this 
information. 

36.  Page  V- 15,  Section  p,  paragraph  4:  We  agree,  and  the  report  has  been  modified  to 
reflect  this  information. 

37.  Page  V-16,  Section  B,  paragraph  3;  We  agree  and  have  modified  the  report  to 
reflect  this  information. 

3  8 .  Page  V- 1 7 ,  Section  B ,  paragraph  6:  Noted. 

39.  Page  V-21  and  Page  V-25,  Tables  V-2  and  V-4:  We  agree  and  have  tried  to  clarify 
the  discussion  in  the  final  report,  in  the  light  of  the  widespread  misunderstanding. 
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40.  Page  V-21,  Section  b(l),  paragraph  2:  Noted. 

41.  Page  V-28,  Section  c.  figure  V-6:  We  discuss  this  issue  in  the  section  on  the 
REMIS  test  in  Chapter  V  and  in  Appendix  D. 

42.  Page  V-29.  Section  CIc,  paragraph  3:  Noted. 

43.  Page  V-32,  Section  c(3),  paragraph  5:  Noted. 

44.  Page  V-36,  Section  f,  paragraph  2;  We  have  revised  the  report  to  make  it 
consistent. 

45.  Page  V-38,  Section  a,  paragraph  2:  The  issue  here  is  ease  of  use  not  functionality. 

46.  Page  V-38,  Section  a,  paragraph  3;  We  agree  and  the  report  has  been  modified. 

47.  Page  V-39,  Section  a,  paragraph  4:  The  issue  here  is  ease  of  use,  not  functionality. 

48.  Page  V-47,  Section  a,  paragraph  2  and  Page  V-49,  Section  e,  paragraph  1:  We 
agree  and  have  modified  the  report  to  reflect  this  information. 

49.  Page  V-49,  Section  d,  paragraph  2:  Noted. 

50.  Page  V-52,  Section  D2,  paragraph  1;  Noted. 

5 1 .  Page  V-53,  Section  2,  paragraph  3:  There  is  a  significant  difference  of  opinion 
between  the  Air  Force  and  DRC  about  how  many  DIREPs  were  related  to 
software.  The  IDA  team  has  no  objective  information  that  all  DIREPs  have  been 
corrected,  and  even  if  we  did,  a  cost  must  be  attached  to  that  process  (whether  it 
has  been  completed  or  will  be  in  the  future). 

52.  Page  V-54,  Section  D3,  paragraph  4:  We  appreciate  the  information.  We  have 
expanded  our  discussion  of  security  for  all  the  systems. 

53.  Page  V-54,  Section  D4,  paragraph  2:  We  considered  the  TICARRS  experience  in 
estimating  costs  of  future  expansions. 

54.  Page  vI-4,  Section  C,  bullet  3:  We  agree  and  have  modified  the  report  to  reflect 
this  information. 
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RESPONSE  TO  IDA  "DRAFT"  REPORT  -  GENERAL 


Specific  Comments 

1.  Page  1-2,  Section  a,  paragraph  7 

All  relevant  wing  personnel  were  not  trained  in  the 
direct  use  of  TICARRS.  Most  supervisors  and  officers  did 
not  attend  any  of  the  training,  even  though  they  were 
asked  to  participate  in  the  surveys  included  as  part  of 
this  report. 

2.  Page  II-8,  Section  2,  paragraph  1 

If  REHIS  requires  a  personal  computer  for  access,  it  is 
^impossible  for  every  Air  Force  user  to  have  the  ability 
to  access  REMIS  data,  since  there  are  relatively  few 
personal  computers  on  each  CAMS  base. 

3.  Page  II-9,  Section  Cl,  paragraph  1 

This  paragraph  states  that  TICARRS  has  a  deployable 
capability,  but  this  is  not  reflected  in  other  tables  or 
capabilities  in  the  report.  Based  upon  'eguirements 
provided  by  the  Air  Force,  DRC  developed  a  deployable 
capability  in  support  of  the  F-117A  for  Desert 
Shield/Storm.  The  deployable  function  enables  the  user 
to  collect  and  view  maintenance,  flight  scheduling,  and 
flight  debriefing  information  using  a  portable 
microcomputer  in  a  stand-alone  environment.  Software 
routines  extract  data  from  the  mainframe  computer  and 
transfer  it  to  the  PC  to  provide  validation  criteria  and 
historical  information  for  the  remote  data  entry 
sessions.  Communications  software  provides  the 
capability  to  automatically  download  to  the  mainframe 
when  a  communications  circuit  is  available. 

4.  Page  II-ll,  Section  2,  paragraph  2 

Field  representatives  were  in  place  as  each  base  was 
activated  on  CDS.  Each  of  the  first  three  bases,  Hill, 
MacDill,  and  Nellis,  had  a  field  representative  when  they 
were  activated  in  February,.  1982. 

The  F-15E  was  not  activated  until  1989. 

TICARRS  began  to  receive  organizational-level  information 
from  CAMS  in  August  1988  at  Luke  AFB,  AZ.  The  conversions 
from  on-line  TICARRS  to  CAMS  was  not  completed  until 
September  1991.  Torre jon  AB,  Spain  remained  an  on-line 
TICARRS  base  until  the  base  was  de-activated  in  March 
1992. 


V-23 


5.  Page  ZI-11,  Section  2,  paragraph  3 

TICARRS  provides  adhoc  query  capability  through  the  PLP 
subsystem  of  IDS  II. 

6.  Page  11-12,  Section  3c,  paragraph  1 

DRC  is  developing  a  Squadron  Centered  Logistics  System 
(SCLS)  and  Repair-Centered  Logistics  System  (RCLS),  both 
of  which  are  to  satisfy  the  initial  requirements  of  the 
systems  of  the  future  as  envisioned  by  this  report. 

7.  Page  III-2.  Section  1,  paragraphs  2,3,4 

Admittedly,  developing  a  framework  of  functions  for 
comparison  of  two  data  systems  is  not  easy.  The 
framework  chosen  lends  bias  to  the  study  in  that  it  is 
basically  a  statement  of  the  existing  capabilities  (with 
two  exceptions)  of  CAMS/REHIS  with  an  additional  column 
^containing  the  capabilities  of  TICARRS,  where  they  happen 
to  coincide.  This  suggests  another  table  might  be  drawn 
conversely,  showing  TICARRS  as  having  most  of  the  blocks 
filled,  but  that  would  not  be  too  helpful  either.  A  more 
evenhanded  table  is  shown  in  Table  v-1.  System  Function 
Comparison,  Attachment  1,  where  features  demanded  and 
used  by  the  field  are  shown.  It  should  be  emphasized 
that  all  the  functions  of  TICARRS  came  about  from 
customer  demand  when  the  customers  had  to  pay 
specifically  for  the  data  system  operation  plus  adding 
features  out  of  their  programs.  None  has  been  free 
gratis  to  the  users. 

8.  Page  III-3,  Section  2,  paragraph  2 

TICARRS (SDS)  supported  trainers  for  the  F-117A  weapon 
system. 

9.  Page  III-5,  Section  a,  paragraph  1 

Although  the  TICARRS  FD  does  not  specify  mathematical 
precision,  the  size  and  randomness  of  the  sample  is  what 
determines  the  accuracy.  The  TICARRS  sample  size  is 
large  enough  to  accurately  predict  any  of  the 
calculations  to  the  same  level  of  CANS/REMIS.  Unless 
otherwise  instructed,  the  computer  carries  out 
single-precision  computations  to  seven  significant 
figures  but  the  reports  rounds,  for  example  for  NTBF,  to 
hours  and  tenths  of  hours. 

-  TICARRS  mathematical  calculations  can  be  accomplished 
100%  accurate  to  four  positions  of  the  decimal  point 
if  the  user  specifies  it  as  a  requirement. 

-  Data  is  always  edited  for  100%  of  the  criteria 
specified. 

-  Transmitted  data  undetected  error  rate  is 
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2.5  X  10(-8)  and  is  automatically  corrected  by  the 
SPRINT  network. 

-  TICARRS  does  all  the  REMIS  specified  Data  Validity 
edits,  except  that  for  data  entered  on-line,  all 
edits  are  accomplished  at  the  source. 

-  Statistical  confidence  limits  have  been  calculated 
and  displayed  for  output  products  in  the  past  and 
could  be  in  the  future  whenever  the  user  specified  it 
as  a  requirement.  90%  confidence  bounds  were 
calculated  for  NTBF  and  MHH/FH  for  the  F-16  until  the 
weapon  system  reached  a  total  of  100,000  flying 
hours . 

10.  Page  III-9,  Section  5b,  paragraph  4 

Over  the  years,  ORC  has  gained  considerable  experience 
interfacing  with  other  systems  both  in  support  of  TICARRS 
requirements  and  for  other  systems.  Since  we  are 
^experienced  with  establishing  interfaces,  we  know  the 
questions  to  ask  and  have  developed  workable  approaches 
to  successfully  working  interface  issues.  We  have 
already  been  through  the  learnng  curve,  which  we  agree 
can  be  very  steep  for  software  developers  who  have  not 
been  through  the  process  several  times.  The  elapsed  time 
for  establishing  working  interfaces  can  be  lengthy,  but 
the  manhours  expended  does  not  need  to  be  extensive, 
provided  an  organizations  experience  can  be  dra%m  upon. 

11.  Page  III-IO,  Section  d,  paragraph  3 

The  last  sentence  of  this  paragraph  is  misleading.  It 
leads  the  reader  to  believe  that  any  base-level  user  can 
access  RENIS  on  part  repair  histories.  Only  those  users 
provided  access  through  a  personal  computer  will  have 
access  to  REMIS.  Any  TICARRS  user  has  access  to  part 
repair  histories. 

12.  Page  III-12,  Section  B,  paragraph  3 

The  report  should  also  include  past  costs  and  schedules, 
estimated  and  actual,  in  order  to  provide  an  indication 
of  the  reliability  of  future  estimates  for  cost  and 
schedule . 

13.  Page  lV-6,  Section  E 

The  HOA  (TAB  C),  for  the  assessment  at  SJAFD  should  be 
included  as  an  appendix  for  reference. 


14.  Page  lv-9.  Section  4,  paragraph  2 
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It  should  be  pointed  out  that  a  substantial  percentage  of 
the  FDWW's  were  double  reports  of  the  sane  problen,  e.g., 
the  same  individual  reporting  a  training  problem  twice. 

16.  Page  V-3/4,  Table  V-1 

Table  V-1  (Attached)  compares  TICARRS  functions  with  the 
functions  of  CANS/REHIS. 

17.  Page  V-3,  Table  V-1 

TICARRS  does  provide  the  capability  to  schedule 
maintenance  at  the  Organizational  (O)  level.  The 
scheduling  of  maintenance  is  not  done  the  same  way  it  is 
done  in  CANS,  when  a  pilot  reported  discrepancy  is 
debriefed,  a  job  control  number  is  created  and  the 
discrepancy  is  printed  at  dispatch.  The  dispatcher  has 
the  responsibility,  as  part  of  his  job,  to  prioritize  the 
^maintenance  scheduling  and  schedule  maintenance 
technicians  to  perform  the  maintenance. 


18.  Page  v-3.  Table  V-1 

Under  serialized  part  maintenance  history  and  "Bad  Actor" 
identification,  the  TICARRS  column  should  indicate 
"LRUs/SRUs" . 

19.  Page  V-4,  Table  V-1 

Even  without  an  automated  SBSS  interface,  TICARRS  does 
track  the  status  of  parts  ordered.  However  the  data  must 
be  entered  and  updated  manually. 

20.  Page  V-4,  Table  V-1 

TICARRS  does  provide  cannabilization  management  using 
CON801,  SCN802  which  depicts  all  relative  cannabilization 
management  information,  including  supply  document  number. 
(See  Sample  Screen  Attached) 

21.  Page  V-4,  Table  V-1 

TICARRS  does  support  Shop  Production 

Planning/Scheduling/Control.  By  recording  "Due-ins"  at 
the  Intermediate  (I)  level  shop,  based  upon  removals  at 
the  flight  line,  the  shop  supervisor  has  the  ability  to 
schedule  and  prioritize  the  LRU  production  access  the 
test  stations. 

22.  Page  V-4,  Table  V-1 

TICARRS-92  has  a  deployable  information  system.  (See 
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Comment  3 ) 

23.  Page  v-5,  Section  3a,  paragraph  1 

It  is  confusing  to  incorporate  the  transfer  of  an 
aircraft  from  one  base  or  squadron  to  another,  as  is 
noted  in  the  paragraph,  with  the  transfer  inspection. 

As  the  report  states  TICARRS  moves  assets  by  recording  a 
change  in  ownership.  The  scheduling  and  recording  of  the 
actions  required  for  a  transfer  inspection  was  quite 
cumbersome  in  TICARRS-92  during  the  operational 
assessment  at  SJAFB.  However,  an  improvement  has  been 
identified  and  will  be  made,  based  upon  an  FOWW  submitted 
by  and  with  the  approval  of  the  Air  Force,  to  make  the 
scheduling  of  all  maintenance  actions  a  much  easier 
process  in  TICARRS-92. 

24.  Page  V-6,  Section  3c,  paragraph  1 

^The  listing  of  primary  users  of  utilization  data  should 
also  include  Plans  &  Scheduling  because  of  the  large 
number  of  inspections  based  on  hours. 

25.  Page  V-6,  Section  3c,  paragraph  2 

The  last  sentence  in  this  paragraph  is  incorrect. 

TICARRS  sortie  history  for  the  month  can  be  found  in 
several  places.  CON570,  Aircraft  History,  provides 
sortie  history  by  equipment  ID  for  any  specified  date 
range.  CON526,  Equipment  Status  &  Utilization,  provides 
sortie  history  by  organization  for  specified  date  ranges. 
An  example  (Attachment  3)  is  shown  for  the  334th  FS  at 
SJAFB  for  the  period  1  Nay  93  through  30  Hay  93.  In 
addition,  TICARRS  SORT-SUM  query  will  retrieve  sortie 
histories  for  periods  up  to  a  year. 

26.  Page  V-8,  Section  3g,  paragraph  3 

In  TICARRS,  the  supervisor  has  the  capability,  during 
supervisor's  review,  to  close  the  job  without  requiring 
an  install  action.  Also,  TICARRS  does  allow  one  work 
center  to  create  work  for  another  work  center  via  CON 
530,  On-Equipment  Maintenance  Reporting. 

27.  Page  V-8,  Section  3g,  paragraph  6 

This  paragraph  fails  to  mention  that  the  4th  FW  at  SJAFB 
did  not  use  a  standard  CAMS  ARC  query  for  their  14  day 
records  check.  TICARRS  does  have  an  ARC  query  used  to 
perform  a  14  day  records  check.  At  SJAFB,  the  TICARRS-92 
ARC  query  did  not  agree  with  the  CAMS  ARC  query  for  a 
particular  aircraft.  The  mismatch  was  quickly  tracked  to 
the  fact  that  four  serial  numbered  items  on  that 
particular  aircraft  had  not  been  linked  in  the  TICARRS-92 
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database  because  they  had  duplicate  serial  numbers  which 
were  transferred  from  CANS  to  TlCARRS-92.  It  should  have 
been  the  responsibility  of  the  crew  chief  to  determine 
the  correct  serial  number  of  the  units  on  his  aircraft 
and  link  those  units  in  the  TICARRS-92  database. 

28.  Page  V-9,  Section  3g,  paragraph  7 

TICARRS  does  allow  the  recording  of  narrative  as  part  of 
an  inspection.  This  is  an  important  feature  of 
TICARRS  which  is  satisfied  by  CON530  and  CON201,  both 
having  unlimited  narrative  space. 

29.  Page  V-10,  Section  3g,  paragraph  12 

TICARRS  PQDR  capability  is  being  updated  to  the  latest 
revision  of  T.O.  00-35D.  However,  the  current  PQDR 
capability  is  being  used  at  Air  National  Guard  and  Air 
^Force  Reserve  units  without  difficulty  and  provides  the 
'^ability  to  handle  returns  to  the  depot  for  credit. 

Per  the  Assessment  Report  published  by  the  Air  Force 
Assessment  Team,  PQDR  was  not  previewed  during  the 
TICARRS-92  preview  at  SJAFB  because  the  capability  would 
be  the  same  as  that  existing  in  TICARRS-87  and  wing 
personnel  were  satisfied  with  that  capability.  However, 
when  the  actual  on-line  data  entry  period  began,  although 
the  capability  remained  the  same,  the  system  could  not  be 
used  because  it  did  not  meet  their  requirements. 

CAMS  includes  a  feature  providing  the  screening  point 
(QA)  with  the  ability  to  return  the  PQDR  to  the 
originator  for  modification  or  possible  deletion.  When 
the  TICARRS  PQOR  capability  was  designed,  both 
maintenance  technicians  and  QA  personnel  were  on  the 
design  team.  The  QA  personnel  only  wanted  the 
technicians  to  make  the  initial  entries;  they,  QA,  would 
take  it  from  there  stating  that  it  was  their  job  to 
ensure  that  it  was  correctly  filled  out  before  it  was 
submitted.  Only  the  Action  Point  (at  an  ALC  or  SPO)  can 
close  a  PQDR. 

30.  Page  V-10,  Section  3h,  paragraph  2 

The  last  sentence  should  be  deleted,  unless  the  report 
goes  into  a  discussion  of  the  issue  of  overall  depot 
level  reporting. 

31.  Page  V-11,  Section  i,  paragraph  2 

To  say  that  TICARRS-92  has  no  CEMS  maintenance  reporting 
function  is  inaccurate.  TICARRS-92  demonstrated  its 
ability  to  collect  and  track  all  engine  mantenance 
information  required  within  the  Engine  Maintenance  Shop. 
TICARRS-92  did  not  have  the  ability  to  interface  this 
data  to  CEMS,  the  responsibility  of  the  Engine  Manger 
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Shop.  (per  the  HOA,  CEHS  interface  was  not  to  be  tested) 


32.  Page  V-11,  Section  j,  paragraph  1 

This  section  deals  more  with  SBSS  than  maintenance.  SBSS 
was  NOT  to  be  evaluated  at  SJAFB,  per  the  HOA. 

33.  Page  V-11,  Section  j,  paragraph  2 

TICARRS  does  provide  an  audit  trail  of  cannibalizations 
when  parts  are  cross-cannibalized.  Each  cannibalization 
does  require  the  opening  of  a  new  wor)c  order,  but  that  is 
In  Accordance  With  (lAW)  proper  Air  Force  documentation 
procedures.  Only  the  document  number  should  move  with 
the  part,  not  the  JCN.  TICARRS  does  support  engine  to 
engine  CANNS  via  CON800. 

^34.  Page  V-13,  Section  ck,  paragraph  2 

When  initially  inputting  TCTO  data  into  CAMS,  the  load 
screen  has  a  data  field  that  asks  if  kits  are  to  be 
ordered.  Response  is  Yes/No.  It  must  be  noted  that  this 
procedure  has  not  worked  for  over  one  year  and  the 
process  is  manually  accomplished  via  AF  Form  2001  to  base 
supply.  However,  kit  status  can  be  monitored  in 
TICARRS-92  via  CON213,  SCN214  and  CON645,  SCN650. 

35.  Page  V-14,  Section  p,  paragraph  2 

TICARRS-92  does  have  a  deployable  information  capability. 
(See  Comment  3}  DRC  was  able,  independently  from  the 
F-117A  and  the  Air  Force  process,  to  procure  a 
communications  link  to  the  Gulf,  Qatar,  in  order  to 
support  the  F-16  unit  deployed  from  Torrejon  AB,  Spain, 
the  last  on-line  direct  reporting  base  to  TICARRS. 
Authorization  to  connect  to  the  line  was  denied  by  HQ 
TAC/LGQ. 

36.  Page  V-15,  Section  p,  paragraph  4 

TICARRS  does  support  a  deployment  capability,  and 
successfully  supported  the  F-117A  during  Desert 
Shield/Storm. 

37.  Page  V-16,  Section  B,  paragraph  3 

TICARRS  (SDS)  Supported  trainers  for  the  F-117A. 


38.  Page  V-17,  Section  B,  paragraph  6 

The  list  on  page  17  appears  to  be  extensive  but  support 
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for  those  functions  will  be  provided  by  enhancements 
being  made  to  TICARRS-92. 

39.  Page  V-21  and  Page  V-25,  Tables  V-2  and  v-4 

When  comparing  the  daily  transaction  data  in  these 
tables#  the  reader  will  notice  the  difference  between 
CAMS  at  12,188  (a  one  shift  average)  and  TICARRS  at 
40,660.  Doubling  the  CAMS  number  to  allow  for  the 
typical  two  shift  maintenance  schedule  gives  a  daily  rate 
of  24,376,  giving  a  ratio  of  CAMS  transactions  to  TICARRS 
of  about  60%.  Over  the  years  both  DRC  and  HQ  Air 
Mobility  Command  (AMC)  (Ref.  Michael  Creasy,  LGXA,  Air 
Mobility  Command,  Scott  AFB,  IL  618-256-563^  have 
observed  that  when  a  base  converts  to  CAMS,  the  number  of 
reported  transactions  drops  to  about  50%  of  prior  levels. 

40.  Page  V-21,  Section  b(l),  paragraph  2 

'During  the  fourth  week  of  the  SJAFB  assessment,  a  team 
from  Gunter  AFB  balanced  the  workload  of  the  DCP's  at 
SJAFB  which  drastically  improved  the  response  time.  DRC 
did  not  make  any  workload  balance  changes  at  the  central 
computer . 

41.  Page  V-28,  Section  c,  figure  V-6 

It  should  be  explained  that  the  three  modules  of  REMIS 
do  not  interact  with  each  other  and  the  impact  that  has 
on  the  users  in  reviewing  a  write-up  that  contains  all 
three  types  of  information. 

42.  Page  V-29,  Section  Clc,  paragraph  3 

There  are  only  two  instances  where  users  may  enter  data 
which  TICARRS  considers  to  be  "in  error".  A  maintenance 
technician  may  report  the  removal  or  installation  of  a 
part  which  TICARRS  considers  to  be  not  installed  on  that 
item  or  not  available  for  installation  because  it  is 
installed  elsewhere  or  is  showing  an  overdue  inspection. 
In  both  instances  the  technician  is  informed  of  the 
perceived  error  condition  and  allowed  to  override.  The 
supervisor  is  then  required  to  ensure  that  the  "error" 
condition  is  cleared  in  TICARRS  before  the  maintenance 
action  can  be  approved. 

43.  Page  V-32,  Section  c(3),  paragraph  5 

TICARRS  has  an  integrated  database  system,  meaning  each 
data  element  is  only  stored  in  one  place  and  can  be 
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accessed  for  any  display.  The  only  instance  of  a 
requirement  to  reenter  data  into  TICARRS  when  moving  to  a 
new  screen  would  be  when  the  data  element  required  for 


V-30 


the  new  screen  was  not  entered  on  the  previous  screen. 
TICARRS  prefills  any  information  required  to  be  moved 
from  one  screen  to  the  next. 

44.  Page  V-36,  Section  f,  paragraph  2 

There  is  an  inconsistency  in  this  paragraph  because 
earlier  in  the  report  it  was  stated  that  TICARRS  would 
require  3x  its  current  power  and  it  is  4.5x  in  this 
paragraph.  ORC  agrees  with  the  requirement  of  3x  the 
current  power. 

45.  Page  V-38,  Section  a,  paragraph  2 

Add  after  ...test  at  Ogden.  TICARRS  suports  "bad  actor" 
program  by  retaining  part/serial  history  for  LRU/SRU  at 
all  reporting  levels. 

^46.  Page  V-38,  Section  a,  paragraph  3 

Recommend  the  last  sentence,  "REHIS  works  better  for  the 
logistics  community",  be  modified  to  state,  "better  than 
_ ",  and  enter  the  reference. 

47.  Page  V>39,  Section  a,  paragraph  4 

Add  a  "bullet"  to  Contractors  section: 

-  TICARRS  LRU/SRU  "bit/piece  history  available  for 
analysis  and  evaluation  by  engineering  for  TCTO  mod 
updates. 

48.  Page  V~47,  Section  a,  paragraph  2  and  Page  v>49, 
Section  e,  paragraph  1 

There  is  an  inconsistency  in  these  two  paragraphs. 
Paragraph  2  on  page  V-47  states,  ..."ACC  personnel 
reported  inaccurate  data  being  drawn  from  EinSURS". 
Paragraph  1  on  page  V-49c  states,  "EinsURS  Data.  REMIS 
seems  to  provide  accurate  current  data  on  mission  capable 
rates,  possessed  hours,  and  other  inventory,  utilization, 
and  status  indicators".... 

49.  Page  V-49,  Section  d,  paragraph  2 

The  loss  of  TCTO  data  is  not  a  new  occurrence.  This  has 
been  on-going  since  the  conversion  of  the  first  F-16  base 
to  CAMS  in  1988. 


50.  Page  V-52,  Section  D2,  paragraph  1 
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Refer  to  response  42  above.  Only  certain  configuration 
errors  are  allowed  to  exist  until  the  supervisor  clears 
them. 

51.  Page  V-53,  Section  2,  paragraph  3 

Of  the  149  Difficulty  Reports  (DIREPS)  generated  during 
the  Operational  Assessment,  about  30  percent 
(approximately  47)  were  categorized  as  software 
deficiencies,  some  of  which  led  to  data  inaccuracies. 
Another  20  percent  (approximately  32)  were  categorized  as 
data  related  problems  with  either  missing  or  incorrect 
data  sourced  from  CANS.  All  DIREPS  reported  were 
corrected  during/or  within  two  weeks  after  the 
Operational  Assessment. 

52.  Page  V-54,  Section  D3,  paragraph  4 

Security  procedures  already  exist  in  TICARRS  to  prevent 
^users  from  entering  data  on  equipment  which  they  do  not 
own.  It  does  not  matter  what  weapon  system  is  involved 
or  what  group  in  the  Air  Force  has  the  requirement.  Each 
TICARRS  user  is  assigned  an  operator  ID  and  password  that 
allows  basic  access  to  the  system.  This  is  the  most 
basic  level  of  security  which  prevents  unauthorized  users 
from  accessing  the  system  at  all.  Granting  of  an 
operator  ID  and  password  alone  will  not  allow  the  user  to 
go  beyond  the  TICARRS  Main  Menu.  Associated  with  each 
operator  ID  are  three  security  elements,  a  training 
indicator,  a  set  of  security  levels,  and  an  authorized 
organization  level.  The  training  indicator  allows 
TICARRS  to  permit  a  user  to  access  any  function,  but 
prevents  any  updates  from  occurring  to  the  database. 

This  is  primarily  assigned  to  support  training  activity 
or  for  DRC  maintainers  to  perform  tests  or  investigations 
on  the  production  system  without  danger  of  corrupting 
data.  The  security  levels  are  used  to  permit  or  restrict 
access  to  the  various  functions  within  TICARRS.  Security 
levels  are  defined  as  a  series  of  "buckets”  with  a 
numeric  level  assigned  to  each.  A  "zero”  in  a  bucket 
indicates  no  authority  for  that  level,  a  "nine”  in  a 
bucket  is  the  highest  authority  for  that  level.  TICARRS 
consists  of  a  nu^er  of  "conversations”  which  support  one 
or  more  functions.  For  example,  CON530  (Maintenance 
Reporting)  provides  only  one  function,  entry  of 
On-Equipment  Maintenance.  CON317  (Part  Pprial  Updates) 
provides  many  functions,  updating  part  numbers,  updating 
serial  numbers  and  recording  inspection  and  time  change 
data  for  serial  number  -  among  others.  Each  function  is 
assigned  a  specific  security  level.  In  the  above 
example,  CON530  is  assigned  a  level  of  3  in  bucket  1. 

This  allows  any  user  with  a  level  of  3  (or  higher)  in 
bucket  1  to  gain  access  to  CON530  and  to  report 
On-Equipment  maintenance.  Because  it  provides  multiple 
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functions,  CON317  is  assigned  multiple  security  levels. 

A  level  of  3  (or  higher)  in  bucket  2  might  allow  the  user 
to  update  the  inspection  or  time  change  data.  A  level  of 
5  (or  higher)  in  bucket  2  might  allow  the  user  to  add  a 
new  serial  number  (this  would  also  allow  access  to 
updating  inspection  or  time  change  data).  A  level  of  7 
in  bucket  might  allow  the  user  to  add  a  new  part  number. 
The  authorized  organization  is  used  to  restrict  users  to 
updating  data  that  "belongs"  to  them.  TICARRS  maintains 
an  organization  structure  from  fleet  level  down  to  work 
center  level.  User's  can  be  authorized  at  various  levels 
within  the  structure,  which  allows  them  to  update  data 
for  that  level  and  all  subordinate  organizations. 

Although  a  user  may  have  the  security  level  required  to 
access  CON530,  s/he  would  not  be  allowed  to  report 
maintenance  data  for  other  organizations.  Another  example 
of  organizational  security  is  the  following:  Users  A  and 
B  are  assigned  the  required  security  level  for  aircraft 
status  changes.  User  A  is  authorized  for  the  4th  FW  and 
''user  B  is  authorized  for  the  334th  FS  (which  belongs  to 
the  4th  FW).  User  A  would  bs  allowed  to  change  the 
status  of  any  aircraft  assigned  to  the  wing  and  User  B 
would  only  be  allowed  to  update  the  status  of  aircraft 
that  belonged  to  his/her  squadron.  In  general,  users  who 
are  authorized  to  access  TICARRS  are  permitted  to  view 
any  output  products  within  the  functions  they  are 
authorized  for.  A  user  with  access  to  CON560 
(Configuration  Management)  could  call  up  a  list  of 
equipment  belonging  to  any  organization  (they  »ould  not 
be  allowed  to  update  the  list  of  that  other  organization 
due  to  organizational  security).  There  are  instances 
where  users  require  access  to  certain  data,  but  are 
restricted  to  certain  data  values.  For  example,  many 
repair  contractors  have  been  allowed  access  to  the 
TICARRS  database.  They  often  need  to  call  up  a 
maintenance  history  of  an  LRU  they  receive  for  repair. 
TICARRS  allows  for  restricting  the  contractor  to  viewing 
history  for  only  LRUs  that  they  are  responsible  for  (thus 
preventing  a  competitor  from  accessing  information). 

They  could  be  restricted  to  data  for  specific  part 
numbers  or  work  unit  codes.  The  TICARRS  security 
structure  provides  the  flexibility  to  support  any  user 
profile.  Security  has  been  designed  into  the 
architecture,  which  means  that  as  each  new  function  is 
developed  and  implemented,  security  capabilities  are 
built  in  without  additional,  costs.  Since  security 
levels,  organizations,  and  data  element  lestiictions  are 
stored  in  the  TICARRS  database,  DRC  is  able  to  provide 
maximum  flexibility.  A  user  can  be  granted  or  denied 
access  to  a  function  (or  the  system)  in  a  few  seconds. 
All  users  can  be  restricted  from  a  specific  function  in  a 
similar  manner  (by  assigning  a  new  level  to  the 
function) . 


V-33 


53.  Page  V-54,  Section  D4 ,  paragraph  2 

TICARRS  has  a  history  o£  successfully  handing  expansion 
throrighout  its  lifetime.  TICARRS  started  with  three 
bases  on-line  with  F-16  A/B  models  and  4  types  of  test 
stations.  TICARRS  has  handled  major  enhancements  and 
continuous  activation  of  units  as  the  F-16s  were  deployed 
worldwide.  During  that  time,  TICARRS  also  handled  the 
expansion  to  two  additional  weapon  systems  on  time  and 
within  budget. 

54.  Page  VI-4,  Section  C,  bullet  3 

TICARRS  already  has  communications  established  to  each  of 
the  RPCs. 
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ATTACHMENT  1 


'Hible  V-1.  System  Function  Comparison 


FUNCTIONS  COMPARED 


CAMS 


REMIS 


nCARRS 


a.  Equipment  lnv«iitor7 


Identification 


Lx>cation 


Assigned/Possessed 


Transactions 


History  -  base  level 


History  -  Command,  theater,  fleet 


Equipment  Transfer  Automated 


Asset  Location 


b.  Equipment  Status 


Current  -  base 


Current  -  Command,  theater,  fleet 


History  •  base 


History  •  Command,  theater,  fleet 


Current  •  Command,  theater,  fleet 


History  -  base 


History  •  Command,  theater,  fleet 


d.  Flight  Scheduling 


e.  Support  MOC 


Automated  MOC 


f.  "Debriefing ' ' 


Flight  profile,  data,  discrepancies 


Work  order  generation 


History  -  base 


History  •  Cotrunand,  theater,  fleet 


Repeat/Recur  Auto  Notification 


g.  Maintenance  Scheduling/Reporting 


Work  order  generation 


Maintenance  planning  &  scheduling 


Job  tracking 
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FUNCTIONS  COMPARED 


TCI  SchedulingAracking 


Phase  Inspection 


FaUure  histories 


Corrective  action  histories 


Reliability  Measurement 


Maintainability  Measurement 


Serialized  part  nuintenance  history 


"Bad  Actor”  IdentiGcation 


Quality  Control/Quality  Assurance 


Fault  Isolation  Help 


Work  Cards  for  Phase 


CND/RTOK 


fa.  ;Mauite3unce<Supply  System  Interfaa 


Job  site  parts  ordering 


Interface  with  configuration  file 


Track  status  of  parts  order 


MICAP,  AWP  management 


Comprehensive  Engine  Management 


Tracking 


Management 


k.  Configuration  IVacking/Management 


1.  TCTO  Management 


m.  Personnel  Thiining/Availability/Scheduling 


n.  Shop  Production  Planning^ScheduIing/Control  X 


o.  Mobilization  Planning 


p.  System  Deployability  i 


Deployable  Communications  Links 


Deployable  Information  System 


CAMS 


REMIS 


nCARRS 


Other  Features 


Test  Station  Tracking 


Status 


Utilization 


Maintenance 


FUNCTIONS  COMPARED 


Trend  Analysis 


'\^rranty  Tracking 


Support  Spares  Procurement 


Blodc  Number  Tracking 


Integrated  Database  Management  System 


Journalization  Process 


E-Man 


Contractor  Network  Interface 


Depot  Maint  Item  Management 


Real  Time  'VTsibOity  of  Fleet  Assets 


*Not  fielded  in  testing 


CAMS 


REMIS 


nCARRS 
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ATTACHMENT  2 


C0N801  5CN802  CVKAG 
Dl$Tl 

ACTIUN 


F530  930721(202)1136 
ACTION 


DONOR 

CANN  TAKEN 

EQUIP 

MAINT 

TAKEN 

DOCUMENT  « 

EQUIP 

OCN 

T 

U 

WUC 

CANN  TC 

1  OCN 

DUCK 

WUC  P/Q 

T/U 

01 

J301AA3 1208306 

4A0300 

1208060 

Y 

H 

B2AA0 

4A0195 

1205404 

OOOO 

82AA0 
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D.  MOA  FOR  THE  OPERATIONAL  ASSESSMENT 

A  copy  of  the  memorandum  of  agreement  (MOA)  for  the  Operational  Assessment  of 
TICARRS  92  (referred  to  as  ‘Tab  C”  in  DRC’s  general  comments)  is  presented  on  pages 
V-41  and  V-42.  DRC  provided  the  MOA  as  part  of  its  comments. 
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MEMO  OF  AGREEMENT 

OPERATIONAL  ASSESSMENT  OF  TICARRS-92 

11  FEBRUARY  1993 


•  Test  Schedule: 

-  Estimate  2/17/93  •  Start  (Base  selection) 
(Firm  by  2/19/93) 

-  Wedc  of  3/1/93  •  preview  (demo) 

-  3/12/93  •  test  plan  due  •  final 

-  3/15/93 -train (2 weeks) 

-  3/29/93  •  start  test 

-  Test  duration  -  6  weeks 


OPR 

ACC 

ACC/DRC/SPO 

TICARRSPMO 

DRC/TestUnit 

DRC/TestUnlt 


•  Functionality  unable  to  be  tested. 

-  SBSS  interface 

-  CEMS  interface 

-  Personnel/training  subsystems 

-  Automated  forms 

•  TICARRS  -92  functions:  See  Attadiment  1 

-  TICARRS-92  Functional  Capabilities  are  baselined  as  of  11  February  1993. 

<  Definition  of  objectives. 

-  To  determine  if  TICARRS-92  can  support  aircraft  maintenance  operations  using 
the  CAMS/REMIS  functions  as  the  baseline  for  comparison. 

•  The  integrity  of  CAMS/REMIS  data  base'  for  the  test  base  will  be  maintained  for  the 
period  of  the  test.  The  test  will  be  stnictured  so  that  data  entry  for  this  purpose  is 
transparent  to  the  persons  making  tiie  TICARRS-92  data  entry.  Costs  associated  with 
this  effort  will  not  be  used  in  the  test  evaluation. 
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MEMO  OF  AGREEMENT 

OPERATIONAL  ASSESSMENT  OF  TICARRS-92 

(CON’T) 

11  FEBRUARY  1993 


.  If  TICARRS-n  is  iioii-ftmcti<»al  in  a  particular  area,  (e.B,tniiiiiis,CEMS,SBSS,  etc)  die 

costs  associated  with  separate  access  to  and  use  of  these  functions  will  be  measured  and 
reported  as  part  of  the  test 

•  A  preview/demonstradon  will  be  conducted  at  the  host  unit  to  develop  the  procedures  associated 
with  both  noo'funcdons  and  workarounds,  taking  into  account  the  following: 

-  Continued  maintenance  of  the  CAMS/REMIS  data  bases 

-  IVorkarounds  to  those  systems  where  TICARRS>92  is  non-functional 

-  Preview  will  be  on  a  TICARRS-92  F-16  database  -  week  of  3/1/93 

-  Any  procedures  and  workarounds  idendfied  sdll  be  documented  and  become  part  of  the 
test  plan. 

•  Test  Oudine  (see  Attachment  2)  to  be  devdoped  and  conducted  in  accordance  with  AFR  700-4 

•  Software  to  be  baselined  and  frozen  prior  to  beginning  of  test  Any  changes  must  be  agreed  to 
by  the  test  director.  Configuration  control  changes  will  be  strictly  enforced  in  accordance  with 
DRC  Standard  procedures  during  test 

Albert  Rand  Clifford  nail 

Dynamics  Research  Corporadon  CAMS/REMIS  System  Program  Director 
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